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Abstract 

Since the second half of the 20th century, Pontryagin's Maximum Principle has been 

widely discussed and used as a method to solve optimal control problems in medicine, 
robotics, finance, engineering, astronomy. Here, we focus on the proof and on the under- 
standing of this Principle, using as much geometric ideas and geometric tools as possible. 
This approach provides a better and clearer understanding of the Principle and, in particular, 
of the role of the abnormal extremals. These extremals are interesting because they do not 
depend on the cost function, but only on the control system. Moreover, they were discarded 
as solutions until the nineties, when examples of strict abnormal optimal curves were found. 
In order to give a detailed exposition of the proof, the paper is mostly self-contained, which 
forces us to consider different areas in mathematics such as algebra, analysis, geometry. 
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1 Introduction 

The importance of Pontryagin's Maximum Principle as a method to find solutions to optimal 
control problems is the main justification for this work. The use and the comprehension of this 
Principle does not always gather together. The understanding of this Maximum Principle never 
finishes as shows the continuous wide number of references in this topic |2l [HI [161 (13 HSl [Ml HH 
ISTl [521 l53l [Ml [701 [7T1 [72] and references therein. We try to contribute to this process through 
a differential geometric approach. 

When we can interfere in the evolution of a dynamical system, we deal with a control system; 
that is, a differential equation depending on parameters, which are called controls. The way we 
interfere in the control system consists of changing the controls arbitrarily. In optimal control 
problems, the controls are chosen such that the integral of a given cost function is minimized. 
That functional to be minimized can correspond with the time, the energy, the length of a path 
or other magnitude related to the system. 

In general, to find a solution to an optimal control problem is not straightforward. A valuable 
tactic to deal with these problems is to restrict the candidates to be solution through necessary 
conditions for optimality, such as those given by Pontryagin's Maximum Principle. This tech- 
nique is used in a wide range of disciplines, as for instance engineering [28l|30l|38l [55] , aerospace 
|69| . robotics |35l [ST] [62] , medicine jU], economics |l5l[57], traffic flow |36]. Nevertheless, it is 
worth remarking that the Maximum Principle does not give sufficient conditions to compute an 
optimal trajectory; it only provides necessary conditions. Thus only candidates to be optimal 
trajectories are found, called extremals. To determine if they are optimal or not, other results 
related to the existence of solutions for these problems are needed. See |2l [H] jSH |53] for more 
details. 

In 1958 the International Congress of Mathematicians was held in Edinburgh, Scotland, 
where for the first time L. S. Pontryagin talked publicly about the Maximum Principle. This 
Principle was developed by a research group on automatic control created by Pontryagin in the 
fifties. He was engaged in applied mathematics by his friend A. Andronov and because scientists 
in the Steklov Mathematical Institute were asked to carry out applied research, especially in the 
field of aircraft dynamics. 

At the same time, in the regular seminars on automatic control in the Institute of Au- 
tomatics and Telemechanics, A. Feldbaum introduced Pontryagin and his collegues to the 
time-optimization problem. This allowed them to study how to find the best way of pilot- 
ing an aircraft in order to defeat a zenith fire point in the shortest time as a time-optimization 
problem. 

Since the equations for modelling the aircraft's problem are nonlinear and the control of the 
rear end of the aircraft runs over a bounded subset, it was necessary to reformulate the calculus 
of variations known at that time. Taking into account ideas suggested by E. J. McShane in [58j, 
Pontryagin and his collaborators managed to state and prove the Maximum Principle, which 
was published in Russian in 1961 and translated into English j6l] the following year. See pT] 
for more historical remarks. 

Initially the approach to optimal control problems was from the point of view of the dif- 
ferential equations |8l [53] [M] [76] , but later the approach was from the differential geometry 
[21 [211 [m [ZO]- Furthermore, the Maximum Principle is being modified to study stochastic 



control systems |12^ H2] and discrete control systems |29| l39l H3] . Lately, the Skinner-Rusk 
formulation |67| has been applied to study optimal control problem for non-autonomous control 
systems, obtaining again the necessary conditions of Pontryagin's Maximum Principle, as long 
as the differentiability with respect to controls is assumed |9j. This formulation is suitable to 
deal with implicit optimal control problems that come up in engineering problems described by 
the descriptor systems \60\ l6T] . This Principle also admits a presymplectic formalism that gives 
weaker necessary conditions for optimality (TTJ [321 E3] • 

Therefore, it is concluded that Pontryagin's Maximum Principle has had and still has a great 
impact in optimal control theory. The references mentioned show that the research is still active 
as for the understanding and also for the applications of the Maximum Principle. 

A symplectic Hamiltonian formalism to optimal control problems is provided by the necessary 
conditions stated in Pontryagin's Maximum Principle. The solutions to the problem are in the 
phase space manifold of the system, but the Maximum Principle relates solutions to a lift to the 
cotangent bundle of that manifold. Thus, in order to find candidates to be optimal solutions, 
not only the controls but also the momenta must be chosen appropriately so that the necessary 
conditions in the Maximum Principle are fulfilled. These conditions are, in fact, first-order 
necessary conditions and they are not always enough to determine the evolution of all the 
degrees of freedom in the problem. That is why sometimes it is necessary to use the high order 
Maximum Principle 146 1 HH l I49j. But, even when we succeed in finding the controls and the 
momenta in such a way that Hamilton's equations can be integrated to obtain a trajectory on the 
manifold, the controls and the momenta are not necessarily unique. In other words, different 
controls and different momenta can give the same trajectory on the manifold, although the 
necessary conditions in the Maximum Principle will be satisfied in different ways. The momenta 
and the controls determine different kinds of trajectories, which can be abnormal, normal, strict 
abnormal, strict normal and singular. We point out that these different kinds of extremals do 
not provide a partition of the set of trajectories in the manifold, because it may happen that 
a trajectory admits more than one lift to the momenta space so that the trajectory is in two 
different categories. 

For years, abnormal extremals were discarded because it was thought that they could not 
be optimal |4H [68] . The idea was that abnormal extremals were isolated curves and thus it was 
impossible to consider any variation of these curves. However, in [59^ it is proved that there exist 
abnormal minimizers by giving an example in subRiemannian geometry. Furthermore, in |56j the 
strict abnormal minimizers are characterized in a general way, studying the length-minimizing 
problem in subRiemannian geometry when there are only two controls. To be more precise, 
a large enough set with abnormal extremals is given and it contains strict abnormal curves 
that are locally optimal for the considered control-linear system. Here began a new interest 
in the abnormal extremals [31 [5l [6l [201 EH [52]. What makes these extremals more special is 
that the abnormality does not depend on the cost function. Hence, the abnormal extremals 
can be determined exclusively using the geometry of the control system. Thus abnormality and 
controllability must be closely related. In fact, in order to have abnormal minimizers, the system 
cannot be controllable. In control theory, controllability is still one of the properties under active 
research [H [7] and the same happens with abnormality in optimal control theory. Moreover, 
the controllability is related with the reachable set. Thus, as first pointed out in |24| [52] , the 
geometry of the reachable set also helps to characterize the abnormal extremals. 

On the other hand, the cost function is essential to prove that abnormal extremals are 
abnormal minimizers, as pointed out in §3.4[ That is why the existence or non-existence of 
abnormal minimizers is only known for specific control problems, mainly control-linear and con- 
trol-affine systems with control-quadratic cost functions or for time-optimal control problems 
[i [6] US [261 [271 [77]. 
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How the necessary conditions of Pontryagin's Maximum Principle are satisfied determines 
the kind of extremals obtained, in particular, the abnormal ones. That is why the thorough 
proof of the Maximum Principle given here gives insights into the geometric understanding of 
the abnormality. Any chance we have along the report to make a comment about abnormality 
will be made because that might help to characterize strict abnormality in the future. 

In this paper, we go through the entire proof of Pontryagin's Maximum Principle translating 
it into a geometric framework, but preserving the outline of the original proof. All details have 
been carefully proved, making us to go into the details of concepts such as time-dependent 
variational equations and their properties, separation conditions given by hyperplanes and con- 
vexity. All this is included as appendices in order not to disturb the continuous evolution of the 
concepts here given. Nevertheless, we assume some knowledge in differential geometry, such as 
the core chapters of [SI] , differential equations [13 EH HO] , and convexity [TSl ES] • 

The control systems in this report are given by a vector field along a projection, that is 
defined in ^ together with its properties. In the heart of the report there are two big parts 
corresponding with two different statements of Pontryagin's Maximum Principle. In ^ and 
Q it is studied the optimal control problem with both the time interval and the endpoints 
given. If the final time is not given and the endpoints are not fixed but they must be in specific 
submanifolds, then the problem is studied in ^and ^ These four sections have been written 
in an analogous way. First of all, two different but equivalent statements of the optimal control 
problems are given. The so-called extended system is the useful one in ^and ^because the 
functional to be minimized is included as a new coordinate of the system. The last subsection in 
^and ^explains the associated Hamiltonian problem that leads to the statements of Maximum 
Principle. In this way, the proof is just in ^and ^ 

One part of the proof of Pontryagin's Maximum Principle consists of perturbing the given 



optimal curve, therefore we introduce in { 3.3 and §5.2|how this curve can be perturbed depending 



on the known data. Above all, it is important the complete proof of Proposition 3.12, although 
known, to our knowledge, there is not a self-contained proof of it in the literature. 

The appendices contain essential results for the core of the report and also some explanation 
to make clear some well-known ideas related to time-dependent vector fields in Appendix |B| 
the reachable set and the tangent perturbation cone in Appendix [C| 

The study of the time-dependent variational equations treated in Appendix [B] gives a clear 
picture of the flows of the complete lift and of the cotangent lift of a time-dependent vector field 
via Propositions B.l, B.2 B.4 |B.5 These results although known, to our knowledge, have not 



appeared in the literature. 

Appendix [C] devotes to the careful study of the connection between the reachable set and 
the tangent perturbation cone, because the proof of Pontryagin's Maximum Principle suggests 
that all the perturbation vectors generate a linear approximation of the reachable set in some 



sense. That sense will become clear in Proposition C.l which proves a result assumed as true 
in the literature. 

To summarize the main contributions of the paper are: 



The proof of Proposition 3.12 that is useful to prove Pontryagin's Maximum Principle. All 
the proofs of this Proposition in the literature, to our knowledge, are not written carefully 
enough. This Proposition is adapted for Pontryagin's Maximum Principle without fixing 



the final time in Proposition 5.9 



• The complete proof of Pontryagin's Maximum Principle in a symplectic framework as in 
|70] . but here we include all the necessary results and the analytical reasoning, which has 
been sketched in great detail. 
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• The highlight of the properties concerning the abnormal extremals that can be deduced 
from the classical result in 



An analytic result necessary in Pontryagin's Maximum Principle, which is proved in Propo- 
sition 



A. 7 This result is used in [Bl], but without proving it. 



• The intrinsic study of the flows of the complete lift and the cotangent lift of a time-de- 



pendent vector field in Appendix [Bj including the proofs of Propositions B.l B.2 B.4 and 

EH 

• The geometric understanding of the interpretation of the tangent perturbation cone as 
linear approximation of the reachable set in Appendix [C] including the proof of Proposition 

EH 

As for the future and actual research line, we point out that all the effort to elaborate this 
work is being used to enlighten the research, from a geometric point of view, on abnormal and 
strict abnormal extremals in optimal control problems in general [10] , and for mechanical systems 
The study of strict abnormal minimizers impose us to consider different cost functions, 
because only the property of being an abnormal extremal depends exclusively on the geometry 
of the control system. That makes the problem of searching for strict abnormal minimizers much 
harder and the possible forthcoming results will be valid only for determined optimal control 
problems. 

To conclude this introduction, we remark that Pontryagin's Maximum Principle provides 
first-order necessary conditions for optimality. These conditions are not always enough to de- 
termine the controls for abnormal and singular extremals, then high order Maximum Principle 
is necessary jl9]. The Maximum Principle works with linear approximation of the trajectories, 
whereas in the high order Maximum Principle high order perturbations must be considered 
|14^ [T5l H6l HHl SH]. The way to construct the proof is the same as in Pontryagin's Maximum 
Principle, but now the tangent perturbation cones are bigger since not only linear approximation 
of the trajectories are considered. In the same way we have provided a geometric meaning to 
most of the elements in Pontryagin's Maximum Principle, we expect to give a geometric version 
of high order Maximum Principle suggested by [l9] , focusing on abnormality. 

The origin of this report was a series of seminars and talks with Professor Andrew D. Lewis 
during his stay in our Department on sabbatical during the first term of 2005. We tried to 
understand the details of the proof as a way to work on some aspects of controllability and 
accesibility of control systems with a cost function, |24| HI] , and where abnormal solutions are 
in the accesibility sets. 

In the sequel, unless otherwise stated, all the manifolds are real, second countable and C°° 
and the maps are assumed to be C°°. Sum over repeated indices is understood. 



2 General setting 

From the differential geometric viewpoint a control system is understood as a vector field depend- 
ing on parameters. Properties about how the integral curves of differential equations depending 



on parameters evolve are explained in |25l |3T1 HOl |50] and used in ^3.3 and ^5.2 

Let M be a differentiable manifold of dimension m and U he a set in M.^ . Consider the trivial 
Euchdean bundle tt: M xU ^ M. 

Definition 2.1. A vector field X on M along the projection it is a mapping X : M xU ^ 
TM such that X is continuous on M x U , continuously differentiable on M for every u £ U 
and tm o X = tt, where tm '■ TM M is the canonical tangent projection. 
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The set of vector fields along the projection vr is denoted by X(7r). If (y,x^) is a local chart 
at X in M, then locally a vector field X along the projection is given by f^d/dx^, where /* are 
functions defined on V x U. 

Let / = [a,b] C M be a closed interval, (7, u) : / — >■ M x U is an integral curve of X if 



j{t) = X{j{t),u{t)). All these elements come together in Diagram (2.1). 



(7,") 




(2.1) 



M xU 



In other words, X is a vector field depending on parameters in U. In this work, the parameters 
are called controls and are assumed to be measurable mappings u: I ^ U such that Im u is 
bounded. Given the parameter u, we have a time-dependent vector field on M, 



Xi"} : I X M 
it,x) 



TM 



X{x,u{t)). 



(2.2) 



For an integral curve (7, u) of X, it is said that 7 is an integral curve of X^^\ as shown in the 
following commutative diagram: 



I X M — ^ TM 



(2.3) 



(7, Id) 




That is, X^"> o (7, Id) = 7 = X o (7, n) . 

A differentiable time-dependent vector field X has associated the time dependent flow or 
evolution operator of X defined as 



: I X I X M 

{t,s,x) 



M 

^^{t,s,x) 



B.l 



for 



where <I>^ is the integral curve of X with initial condition x at time s. See Appendix 
more details. Moreover, the evolution operator defines a diffeomorphism on M that is used in 



the following section n : M 



As the controls u: I ^ U are measurable and bounded, the vector fields X^"^ are measurable 
on t, and for a fixed t, they are differentiable on M. Hence, the notion of Caratheodory vector 
fields must be considered \25\ |3T] from now on. Then, we only consider absolutely continuous 
curves 7: / — > M to be generalized integral curves of the vector field X^'^^; that is, they only 
satisfy 7 = X o (7, n) at points where 7 is derivable, which happens almost everywhere. The 
existence and uniqueness of these integral curves are guaranteed once the parameter is fixed 
because of the theorems of existence and uniqueness of differential equations depending on 
parameters. For more details about absolute continuity, see Appendix A and j25 l |3 H 174) . 
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3 Pontryagin's Maximum Principle for fixed time and fixed end- 
points 

We particularize the general setting described in ^for optimal control theory. To make clear 
we are in a specific case the manifold is denoted by Q, instead of M. 



3.1 Statement of optimal control problem and notation 

Let Q be a differentiable manifold of dimension m and U <Z M.^ a. subset. Let us consider the 
trivial Euclidean bundle ir: Q x U ^ Q. 



Let X be a vector field along the projection vr : Q x [/ — > Q as in Definition 2.1 If (y,x^) is 
a local chart at a point in Q, the local expression of the vector field is X = f^d/dx^ where /* 
are functions defined onV xU . 

Let / C M be an interval and (7, u) : / — > Q x [/ be a curve. Given F: Q x U — > M, let us 
consider the functional 

Sh,u] = I Fh,u)dt 



defined on curves (7,^) with a compact interval as domain. The function F: Q x {/ — > M is 
continuous on Q x U and continuously differentiable with respect to Q on Q x U. 

Statement 3.1. (Optimal Control Problem, OCP) Given the elements Q, U, X, F, I = 
[a, 6] and the endpoint conditions Xa, xi, € Q, consider the following problem. 

Find (7*,u*) such that 

(1) endpoint conditions: 7*(a) = Xa, 7*(f') = Xf,, 

(2) 7* is an integral curve of X^'^*\- j*{t) = X (j* (t) , u* {t)) , t £ I, and 

(3) minimal condition: S['y*,u*] is minimum over all curves (7, ti) satisfying (1) and (2). 

The tuple {Q, U, X, F, I , Xa, Xf)) denotes the optimal control problem. The function F is called 
the cost function of the problem. The mappings u: I ^ U are called controls. 

Comments: 



1. The curves considered in the previous statement satisfy the same properties as the gen- 
eralized integral curves of vector fields along a projection described in ^ That is, 7 is 
absolutely continuous and the controls u are measurable and bounded. 

2. Locally, condition (2) is equivalent to the fact that the curve (7*, u*) satisfies the differential 
equation i;* = /*. 



3.2 The extended problem 

Taking into account the elements defining the optimal control problem and their properties, we 
state an equivalent problem. 

Given the OCP {Q,U,X,F,I,Xa,Xb), let us consider Q = x Q and the trivial Euclidean 
bundle n: Q x U Q. 

Let X be the following vector field along the projection tt : Q x U ^ Q: 

X{x°,x,u) = F{x,u)d/dx^\(^^o^^^u^ +X{x,u), 
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where is the natural coordinate on M. According to Equation (2.2), this vector field can be 
rewritten as 

Given a curve (7, u) = {{x^ o 7, 7), u) : I ^ Q x U such that 7 is absolutely continuous and 
u is measurable and bounded, the previous elements come together in the following diagram: 




where 7r2 is the projection of Q onto Q. 

Statement 3.2. (Extended Optimal Control Problem, OCP) Given the above-mentioned 
OCP (Q, [/, X, F, I ,Xa,Xf,), Q and X, consider the following problem. 

Find (7*,u*) such that 

(1) endpoint conditions: 7* (a) = {0,Xa), 7*(^) = Xb, 

(2) 7* is an integral curve of X^'^*\- j* {t) = X (j* {t) , u* (t)) , t £ I, and 

(3) minimal condition: "f*^\b) is minimum over all curves (7, u) satisfying (1) and (2). 

The tuple (Q, U, X, I ,Xa,xi)) denotes the extended optimal control problem. 

1. The functional 7*° (6) to be minimized in the OCP is equal to the functional defined in 
the OCP. That is to say, we have 

5[7,n] =70(6) = / F{j,u)dt = Sh,u] 

J a 

for curves (7, u). 

2. Locally, the condition (2) is equivalent to the fact that the curve {j*,u*) satisfies the 
differential equations x^ = -F, = /*. 

The elements in the problem {M,U,X,I,Xa,Xb) satisfy properties analogous to the ones 
fulfilled by the elements in the problem {M,U, X, F, I,Xa,xi)), but for different spaces; see ^ 



^3.1 for more details about the properties. 



3.3 Perturbation and associated cones 

The following constructions can be defined for any vector field depending on parameters — see 
^2] — in particular, for those vector fields defining a control system. In order not to make the 
notation harder, we will construct everything on M, but the same can be done on Af or on any 
other convenient manifold, as for instance the tangent bundle TQ for the mechanical case. 
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3.3.1 Elementary perturbation vectors: class I 

Now we study how integral curves of the time-dependent vector field X^'^^ : M x I ^ TM, 
introduced in ^ change when the control u is perturbed in a small interval. 

In the sequel, a measurable and bounded control u: I = [a,b] ^ U and an absolutely 
continuous integral curve 7: I — > M of X^^J' are given. Let vri = {ti,li,ui}, where ti is a 
Lebesgue time in (a, 6) always for the X o (7, u) — i.e. it satisfies Equation (A. 17) — li G M^, 



til G U. From now on, to simplify, ti is called just a Lebesgue time. For every s G small 
enough such that a < ti — lis, consider u[ttI] : I ^ U defined by 

u\Tr^](t) = / * ^ ~ hs,ti], 

1^ u{t), elsewhere. 

Definition 3.3. The function u[vrf] is called an elementary perturbation of u specified by 
the data vri = {ti,li,ui} . It is also called a needle like variation. 



Associated to ti[vr|], consider the mapping 7[7r|] : I M, the generalized integral curve of 
^^{"['^1]} with initial condition (0,7(0)). 

Given e > 0, define the map 

ip^,:Ix [0, e] ^ M 

{t,s) ! — > 9?7ri(t,s) =7kf](i) 

For every t £ I, ip^^ : [0, e] ^ M is given by (fi^is) = ifn^it, s). 

As the controls are assumed to be measurable and bounded, it makes sense to define the 
distance between two controls u,u: / — > [/ as follows 

d{u,u) = / \\u{t) -u{t)\\ dt 



I 

where || • || is the usual norm in M.^. Here, a bounded control u: I ^ U means that there 
exists a compact set in U that contains Im u. The control u[7rf] depends continuously on the 
parameters s and vri = {^i, h,ui}; that is, given e > there exists 6 > such that if |ti — t2\ < 6, 
\li — h\ < ^1 — U2\\ < S, \si — S2I < then d(n[7rj^], ii[7r2^]) < e. 

Hence the curve c/?^^ depends continuously on s and vri = {ti,h,ui}, then it converges 
uniformly to 7 as s tends to 0. See |25|IHT] for more details of the differential equations depending 
continuously on parameters. 

Let us prove that the curve 99^^^ has a tangent vector at s = 0. Let it[vrf] be an elementary 
perturbation of u specified by vri = {ti, h, ui} and consider the curve ip^^^ : [0, e] — > M, (/7^\(s) = 
7[vrf](ti). 

Proposition 3.4. // ti is a Lebesgue time, then the curve ip^^_^ : [0, e\ ^ M is differentiable at 
s = 0. Its tangent vector is [X{'j{ti),ui) — X{^{ti),u{ti))] li. 

Proof. It is enough to prove that for every differentiable function g: M ^ M, there exists 

^^^.^ g{^i\{s)) - g{^i\m ^ 

s^O s 

As this is a derivation on the functions defined on a neighbourhood of 7(ii), it is enough to 
prove the proposition for the coordinate functions of a local chart at 7(^1). Thus take g = x^, 

^ ^ ^.^ o O(^) - 0(0) ^ ^.^^ (^' ° 7[vrf])(ti) - jx' o 7)(ti) 

s^O S s^O S 

s-fO S 



8 



As 7 is an absolutely continuous integral curve of X^"> , j{t) = X{-f{t),u{t)) at every Lebesgue 
time. Then integrating 

y(ti)-y(a)= r fm,u{t))dt 



and similarly for 7[vrf] and ^i[7rf]. Observe that 7['7rf](t) = 7(t) and u[7r|](t) = u{t) for t G 
[a, ti — lis). Then, 

^ ^ fa n7MKt),u[nm)dt - C fm,u{t))dt 

s^O S 

= lim — - — — . 

s^O S 



As ti is a Lebesgue time, we use Equation (A.17): 

ft 



X{j{s),u{s))ds = hX{j{t),u{t)) + o{h) 

t-h 



in such a way that 



s^O S 

= lim[f(7[vr?](ii),ni)-f(7(ti),^^(ti))]/i. 
As /* is continuous on M, we have 
A = lim[f{j[nl]{h),u,) - f{j{h),u{t,))]h = [/'(lim 7[vr?](ti), ^xi) - f (7(^1), n(ti))] /i 
= [f (7(*i), ni) - r{^{h),u{h))] h = [(X(7(ti), m) - X(7(ti), h] (x^). 

□ 

Definition 3.5. The tangent vector v[ki\ = {X{'^{ti),ui) — X{'^{ti),u{ti))) li ^ T^^^^^-^M is the 
elementary perturbation vector associated to the perturbation data vri = {ti,li,ui} . 
It is also called a perturbation vector of class I. 

Comments: 

(a) The previous proof shows the importance of defining perturbations only at Lebesgue times, 
otherwise the elementary perturbation vectors may not exist. 

(b) Observe that if we change tti = {ti,li,ui} for tt2 = {ti,l2,ui}, then v[Tri] = (h/h) ^[712]. 
If v[Tri] is a perturbation vector of class I and A G M"*", then A v[tti] is also a perturbation 
vector of class I with perturbation data {ti,X li,ui}. 

(c) We write h(w)g for the derivative of the function g in the direction given by the vector 
w G TxM. Due to Proposition |3.4[ for every differentiable function g: M — > M we have 



9{vi\{s))-9{l{ti))-sL{v[7ri])g ^ 

S s^O 

Hence 

g {^i\{s)) = g (7(ti)) + sL{v[Tri])g + o{s). 
If (x*) are local coordinates of a chart at 7(^1), 
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That is, 

{cpi\y{s)=-f\h)+sv[7riY + o{s). 

Now, if we identify the open set of the local chart and the tangent space to M at 7(ti) 
with the same space M™', we write the following linear approximation 

ipi\{s)=^{ti) + sv[7ri] + o{s). (3.4) 

The initial condition for the velocity given by the elementary perturbation vector evolves 
along the reference trajectory 7 through the integral curves of the complete hft (X^)^"^ of 



X^'^\ as explained in Appendix B.2 Note that '/'^^(s) = 'J'^^") for t > ti because of 



the definition of (/^tt^ and ti[vr|]. 

Proposition 3.6. Let V[tii\ \ TM he the integral curve of the complete lift (X^)^"^ of 

with initial condition (ti, (7(^1), f [vri])). For every Lebesgue time t G {ti^h], V['Ki\{t) is the 
tangent vector to the curve 93* : [0, e] ^ M at s = 0. 



Proof. The proof follows from Proposition B.l and the definition of the curves considered. □ 



3.3.2 Perturbation vectors of class II 

The control can be perturbed twice instead of only once, in fact it may be modified a fi- 
nite number of times. If ^2 is a Lebesgue time greater than ti and we perturb the con- 
trol with vTi = and 112 = {i2i^2)'U2}) then we obtain the perturbation data 7ri2 = 
{(ii,i2),(^i,/2),(wi,M2)}, which is given by 

{lil, t£[ti- liS,ti], 
U2, te[t2-l2S,t2], 
u{t), elsewhere 

for every s G M"*" small enough such that [ti — hs,ti] n [t2 — l2S,t2] = 0. Then 7[vrf2] : / — > 
M is the generalized integral curve of X^^t'^izl^ with initial condition (0,7(0)). Observe that 
7^12] (i) = l{t). Consider the curve 99^2 ^ • e] ^ given by if^^^^is) = 7ki2](i2)- 

Proposition 3.7. Let ti, t2 be Lebesgue times such that ti < t2- The vector tangent to 
<y9^2j^ : [0,e] M at s = is v[k2] + V[TTi]{t2), where V[7ri]: [ti,b] — > TM is the generalized 

integral curve of (X-^)^"^ with initial condition {ti, {^{ti),v[TTi])). 

Proof. Here we perturb the control first with tti along 7 and we obtain M[7rf]. Then we perturb 
this last control with the other perturbation data, 7r2, along 7['7rf]. Then the superindeces of 
the tangent vectors denote the curve along which the perturbation is made. As in the proof of 
Proposition |3.4[ 



^ ^ lii^ (^^ ° V>i\,)is) - (x- o ip'^JjO) ^ ^.^ (x- o 7[vrf2])(t2) - (x- ° 7){t2) 

s—>0 S S 

1-^ 7ivrf,](t2) - Yit2) _ ^.^ ( l>t2]it2) - fjntm ^ l^tm - lKt2) 

s-»0 S s^O y s s 

We understand 7['7rf2] as the result of perturbing 7[vr|] with 7r2, and use the linear approximation 



in Equation (3.4) for 7[vrf2](t2) and 7[vrf](t2) according to Proposition 3.4 



^WM = 7[vr?2](i2) = l[<]{t2) + sv['K2V^-'^^ + o{s), 
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7M(t2) = 7(^2) + S V[TTir{t2) + 0{S). 

Then 



s^O \ S si s—>0 \ / 

As 7[vrf] depends on s and s tends to 0, A = L (i'[vr2]''' + V^[vri]'''(t2)) ■ □ 



Considering identifications similar to the ones used to write Equation (3.4), we have 
H^XM = 7(^2) + sv\k2\ + sl^[vri](t2) + o{s). 

Now we define how the control changes when it is perturbed twice at the same time. If t\ 
is a Lebesgue time, vr'^ = {ti, li'^^} and vr" = are perturbation data, then tth = 

{(ti, ti), (/'j^, /"), (ti']^, u'/)} is a perturbation data given by 

[ elsewhere. 

for every s G M"^ small enough such that a < t\ — (/'^^ + l'x)s. Then 7[7rf]^] : / — > M is the 
generalized integral curve of with initial condition (0,7(0)). Observe that 7[vrJ;^](t) = 

7(i). Consider the curve (/?^\^ : [0,e] M, defined by 92^\^(s) = 7[vrf;^](ti). 

Proposition 3.8. Let ti be a Lebesgue time. The vector tangent to (^^'^^^ : [0, e] ^ M at s = is 
v[7r[] + ^[Tr"], where w[vr^] and ^[vr"] are t/ie perturbation vectors of class I associated to vr^ and 
vr", respectively. 

Proof. As in the proof of Proposition |3.4| 

^ ^ ^.^ (x'o(^i\j(5)-(x^oyi\j(o) ^ ^.^ yK](ti)-y(ti) 

As 7 is an absolutely continuous integral curve of 7(t) =X(7(i),'u(t)) at every Lebesgue 

time. Then, we integrate 

y(ti)-y(a)= r f\^{t),u{t))dt 



and similarly for 7[7rf;^] and w[7rfi]. Observe that 7[vrf]^](t) = ^{t) and u[7rj]^](t) = u{t) for 
t G [a,ti - {l[ + l'{)s). Then, 

^ ^ li^ /g^ f (7kfi](t),^[7rfi](t))dt - /^^^ f\^{t)Mt))dt 

= lim — — 

S 



itS^+vos [ni[^'m),u',) - f^m,n{t))j dt 



lim 



Itl^s [fKlK^]{t),n';)-f\^{t),u{t)) 



dt 
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As ti and ti — l'{s are Lebesgue times and a < ti — {l[ + /'/)s for a small enough s, Equation 



(A. 17) is used. Now we have 

s^O I S 

s 

= hm ([f (TklKti - - filih - l'is),u{ti - I'is))] 

+ (f(7K](ti),<)-r(7(ti),n(ti)))/'/)- 
As is continuous on M x [/, we have 



A 



= (f (lmi7[vr;i(ti - l'is),u'^) - f (lim7(ti - /'/s), Ihn - /'/s))) 
+ (f (lim7[vr?i](ti),<) - f (7(ti),n(ti))) = [r(7(ti),n;) - f{j{h),u{h))] l[ 



+ (r(7(ti),<) - f\l{h)Mti))) I'i = L [vW,] + v[K'i\) {x'). 

□ 



Analogous to the linear approximation (3.4 1, we have 



/Ai(^) = 7(ii) + sv[tt[] + + o(s). 

If we perturb the control r times, tt = {tti, . . . , tt^.}, with a < ti < . . . < tj- < b, then 7[vr*](t) 
is the generalized integral curve of X^"''^"!'' with initial condition (a, 7(a)). Consider the curve 
ifi: [0,e] ^ M for t e [tr,b] given by ifUs) = 7[vr1(t). 

Corollary 3.9. For t G [tr,^], the vector tangent to the curve (p\^: [0,e] ^ M at s = is 

V['Ki\{t) + . . . + y[7rr.](t), where ¥[1:1] : [ti, b] TM is the generalized integral curve of (X^)^"^ 
with initial condition {ti, (7(^4), v[vrj])) for i = 1, . . . ,r. 



This corollary may be easily proved by induction using Propositions 3.4, 3.7 3.8 where 
all the possibilities of combination of perturbation data have been studied. If w is the vector 
tangent to y?^ at s = 0, the perturbation data will be denoted by vr^. Bearing in mind the 



different combination of vectors in Definition D.2 we have the following definition. 



Definition 3.10. The conic non-negative combinations of perturbation vectors of class I and 
displacements by the flow of X^^J' of perturbation vectors of class I are called perturbation 
vectors of class II. 

3.3.3 Perturbation cones 

Considering all the elementary perturbation vectors, we define a closed convex cone at every 
time containing at least all displacements of these vectors. To transport all the elementary 
perturbation vectors, the pushforward of the flow of the vector field X^^^ is used. See Appendix 
[B| Observe that the second comment after Definition 3.5 guarantees that the set of elementary 
perturbation vectors is a cone. 
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Definition 3.11. Fort G {a,b], the tangent perturbation cone Kt is the smallest closed con- 
vex cone in T^(i-^M that contains all the displacements by the flow of X^"^^ of all the elementary 
perturbations vectors from all Lebesgue times r smaller than t: 



I 



conv 



U 



. a<T<t 
\t is a Lebesgue time 



where Vr denotes the set of elementary perturbation vectors at r and conv(^) means the convex 
hull of the set A. 

To prove the following statement, we use results in Appendices [P] and [E{ precisely Proposition 



D.4, D.5 and Corollary E.2 



Proposition 3.12. Let t £ (a, b]. If v is a nonzero vector in the interior of Kt, then there exists 
e > such that for every s G (0, e) there are s' > and a perturbation of the control u['k'^] such 
that 7[7r^](t) = 7(t) + s'v. 



Proof. As V is interior to Kt, by Proposition D.5 item (d), v is in the interior of the cone 



conv 



U 



V, 



a<T<t 
\t is a Lebesgue time 



where Vt is the cone of perturbation vectors of class I at time r. Hence, v can be expressed as 



a convex finite combination of perturbation vectors of class I by Proposition D.4 



Let iW, x*) be a local chart of M at 7(t). We suppose that the image of the local chart and 
W are identified locally with an open set of W^. Through the local chart we also identify T^^t-^M 
with M™". We consider the affine hyperplane 11 orthogonal to v at the endpoint of the vector v 
and identify 11 with M'""^ 

A "closed" cone denotes a closed cone without the vertex. Observe that such a cone is 
not closed, that is why we use the inverted commas. We can choose a "closed" convex cone C 
contained in the interior of C such that v lies in the interior of C and {w, > for every w £ C. 
For example, we can consider a circular cone with axis v satisfying the two previous conditions, 
as assumed from now on. Hence 

UnC = v + B{0,R), 



where B{0,R) is the closure of an open ball in the subspace orthogonal to v, denoted by v . 
For r £ v^, we will write r instead of + r as a vector in M™. 

Let us construct a diffeomorphism from the cone C to a cylinder of M™. If u; G C, the 
orthogonal decomposition of w induced by v and is 



w 



{w,v) 



+ w 



{w,v) 
{v,v) 



{w,v) 
{v, v) 



v + 



{v,v) 



-w — V 



Observe that w 

(w,v} 

the vertex, we have the map 

g:C 

w 



f is a vector in B{0,R) C v . Considering the "closed" cone C without 



+ X B{0,R) 

{w,v) {v,v) 
{v,v) ' {w,v) 



w 



{s,r), 
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that is a C°° diffeomorphism with inverse given by 

g-^: M+ X B{0,R) — > C 

(s,r) 1 — > s{v + r)=w. 

Note that g and can be extended to an open cone, without the vertex, containing C, so the 
condition that g is diffeomorphism is clear. 

If we truncate C by the affine hyperplane 11, we obtain a bounded convex set C^. The 
restriction of g to is g^: ^ (0, 1] x -6(0, R), that is also a C°° diffeomorphism with inverse 
g-^ : (0, 1] X B{0,R) ^ 



If r G -6(0, R), then wq = v+r is interior to C. Hence, associated to wq we have a perturbation 
TTy^Q of the control u. Let 7['^«)o] • ^ — ^ ^■'^^ generalized integral curve of X^"["»ol> with 

initial condition (0,7(0)) and consider the map 



T : [0,1] X 5(0, i?) — y M 

^s,r) ^ r(s,r)=7[<o](t) 

(0,r) ^ r(0,r)=7(t), 

which is continuous because 7[vr^p](t) depends continuously on s and vr^^ and 

lim r(s, r) = 7(t) = r(0, ro). 

(s,r)^(0,ro) 

Hence, for every e > 0, there exist 61,82 > such that if |s| < 5i and ||r|| < 82, then ||r(s,r) — 
r(0,0)|| = ||7KJ(t)-7(i)ll<e. 

Taking e such that B{'y{t),e) is contained in W, there exist 61,62 > such that if |s| < 61 
and ||r|| < 62, then 7[7r^J(t) S VF. 

We consider now the map 



A: [0,6i]xB{0,62) T^^t)M ^W- 

{s,r) ^ A(s,r)=7[<J(t)-7(t) 

(0,r) I — > A(0,r)=0 

that is continuous because lim(3 ,.)^(o,ro) ^(•^i^) = = A(0,ro). Remember that we have iden- 
tified W with M*" via the local chart. With this in mind and using Equation (3.4), we can 
write 

7K0] W - 7(*) = + + Or(s), 

where Or{s) S W^. 

We are going to show that, taking (s,r) in an adequate subset, A(s,r) lies in the interior of 
the cone C. 

Take a section of the cone through a plane containing v and w, and compute the distance 
from the endpoint of w to the boundary of the cone C. This is given by 

s{R-\\t\\) 



1 + 



M 



This is the maximum value for the radius of an open ball centered at the endpoint of s{v + r) 
to be contained in C. 

Define the function 



9: [0, <5i] X 5(0, (52) — > 

(0,r) ^ 0. 
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which is continuous because lim(^ ,,)^(o,ro) G>is,r) = = 0(0, ro). Take 

R-62 



1 + 



R 

M 



then there exist di, 82 > such that, if |s| < 61 and ||r|| < 82, then ||0(s,r)|| = ||or(s)/s|| < e. 



If (s,r) E (0,(5i) X 5(0,(52), then 



||A(s, r) — s{v + r)\\ = \\s{v + r) + Or{s) — s{v + r)\\ = ||or.(s)|| < se < s 



R- \\r\\ 



1 + 



R 

M 



since ||r|| < ^2 < ^2 < -R- Thus we conclude that A(s,r) = s{v + r) + Or{s) is in the interior of 
the cone C for every (s, r) G (0, 5i) x i?(0, 52)- 

Now, for s G (0,(5i), we define the continuous mapping 



Gs-. B{0,52) 



B{0,R) C M™"^ 

Gs(r) = (7r2 051 o A) (s,r). 



(3.5) 



where 7T2 : x B{0, R) B{0, R), 712(5, r) = r. Observe that for ro S -6(0, ^2) we have 



lim Gs(r) = lim 

(s,r)^(0,r-o) (s,r)^(0,ro) 



(t;,t;) 



s{v,v) + {o{s),v) 



{s{v + r) + o(s)) — t> 



and 



(5 o A)(s, r) = ^(tKoK*) - 7(i)) = 5(K^ + + Oris)) = {s' , r'). 



(3.6) 



Suppose that there exists r G -6(0, R) such that Gs{r) = 0. Then applying g ^ to (3.6), we have 



A(5,r) = 7KJ(t) - 7(i) = 9-\s',0) = s'v. 



(3.7) 



Hence, to conclude the pro of we need to show that there exists r with Gs{r) = for s small 
enough. To apply Corollary E.2 there must exist r' E -6(0, 82) such that ||Gs(r) — r|| < ||r — r'\\ 



for every r £ d |^-B(0, 82) j ■ We will show that the condition is fulfilled for r' = 0. 
Consider the mapping 



g-. [0,5i]x B{0,62) 
{s,r) 
(0,r) 



5(0, -R) C M'""^ 
^(s,r) = Gs{r) - r 

g{o,r) = o. 



For ro G .6(0,(52), we have lim(s,r)^(o,ro) ^(s; '^) = li™(s,r)^(o,ro) G's(^) - r = 0. Thus G is 
continuous. 



Given tq G 9 |^-6(0, (52) j , take e = S2/2, then there exist (5o(0,ro), (5i(0,ro) > such 
that if |s| < (5o(0,ro) and ||r - ro|| < (5i(0,ro), then \\g{s,r) - ^(0,ro)|| < 62/2. Hence 
^i{0,ro)) \ rQ e d (^-6(0, (52)^ | is an open covering of the boundary of -6(0,(52); 5.6(0,(52). 
As this is a compact set, there exists a finite subcovering. 



{B{n,6i{0,n)),...,B{ri,6{0,n))}. 
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Take 6 as the minimum of {(^o(0, n), . . . , (5o(0, n)}. Let us see that, for every (s,r) € [0,6] x 
dB(0, 62), \\Gs{r) — r\\ < \\r\\. As r is in an open set of the finite subcovering, 

\\g{s,r)\\ = \\Gs{r)-r\\ < ^ <52 = ||r||. 



Hence, using Corollary E.2 for every s G {0,5), the set Gs{B(0,62)) covers the origin; that is. 



there exists r G -6(0, 82) such that 

Gs{r) = {Tr2ogoA){s,r) = 0. 



Then, because of the definition of the mapping Gs in Equation (3.5) and Equations (3.6) and 



(3.7), there exists s' £ M"*" such that 

iKoKt) = lit) + s'v. 

To finish the proof we only need to take tt^ = vr^p. In other words, we have a trajectory coming 
from a perturbation of the control that meets the ray generated by as wanted. □ 

3.4 Pontryagin's Maximum Principle in the symplectic formalism for the 
optimal control problem 

In this section, the OGP is transformed into a Hamiltonian problem that will allow us to state 
Pontryagin's Maximum Principle. 

Given the OCP {Q,U, X, F, I, Xa, Xb) and the OGP {Q,U,X,I,Xa,Xb), let us consider the 
cotangent bundle T*Q with its natural symplectic structure that will be denoted by to. If 
{x,p) = {x^,x,po,p) = {x^,x^, . . . ,x"^,po,pi, . . . ,Pm) are local natural coordinates on T*Q, the 
form u) has as its local expression uj = dx^ A dpQ + dx^ A dpi . 

For each u £ U, : T*Q ^ M is the Hamiltonian function defined by 

m 

H'^ip) = H{p,u) = {p,X(x,u)) = poF{x,u) + ^pif{x,u), 

i=l 



where p G T^Q- The tuple {T*Q,lo,H'^) is a Hamiltonian system. Using the notation in (2.2), 
the associated Hamiltonian vector field y^"}^ satisfies the equation 

i(yW)c^ = dH"". 

Thus we get a family of Hamiltonian systems parameterized by u, H : T* Q x U ^ M., and the 
associated Hamiltonian vector field Y : T*Q x U ^ T{T*Q) which is a vector field along the 
projection vfi : T*Q x [7 — > T*Q. Its local expression is 

Y{p, u) = i^F{x, ^ + / 5^ + 5^ + [-P0 ^ u) - p,—^{x, n) J — J . 



It should be noted that Y = X'^* is the cotangent lift of X. See Appendix B.3 for definition 
and properties of the cotangent lift. 

Given a curve (A,n): / — > T*Q x U such that it is absolutely continuous on T*Q, it is 
measurable and bounded on U , and 7 = vTq o A; if vTq : T*Q — > Q is the natural projection, the 
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previous elements come together in the following diagram: 

T{T*Q) 




•T'Q 



— T*Q X U ^T*Q 




Statement 3.13. (Hamiltonian Problem, HP) Given the OOP {Q,U, X, F, I,Xa,Xb), and 
the equivalent OCP {Q, U, X, I, Xa, Xb), consider the following problem. 

Find {d*,u*) such that 

(1) 7*(a) = (0,Xa) and 7*(6) = x^, if^* = tt^o?*, 7* = tt2 07*. 

(2) d*{t) = X^'{a*{t),u*{t)), t E /. 

The tuple {T*Q,U,X^\I ,Xa,Xf)) denotes the Hamiltonian problem as it has just been de- 
fined and the elements satisfy the same properties as in f|2} 

Comments: The Hamiltonian problem sati sfies analogous conditions to those satisfied by the 



OCP and the OCP defined in ^ 3.1 



and ^3.2 



respectively. 



1. Given {^,u), the function u: I ^ U allows us to construct a time-dependent vector field 
on T*Q, T*Q xl ^ T{T*Q), defined by 

{X^')^^Hx,p,t) = X''\x,p,u{t)). 

Condition (2) shows that a* is an integral curve of (X-^*)^" h 

2. The vector field {X'^ )^"^ is vrg-projectable and projects onto X^^\ Thus if a is an 
integral curve of {X'^*)^^\ 7 = o a is an integral curve of X^^\ 

3. Locally, conditions (1) and (2) are equivalent to the fact that the curve satisfies the 
Hamilton equations of the system {T*M,uj, H'^), 



X 

Po 

Pi 



dH"" _ 
dpo 

(9i/" _ 

dpi 



F 
f 



dx^ 
dH"" 



^ Po = ct 

dF dp 

dx'^ ^■^ dx^ ' 



(3.8) 
(3.9) 



and satisfies the conditions 7(a) = (0,Xa), 7(6) = Xb- 

In the literature of optimal control, the system of differential equations given by Equations 



(3.8), (3.9) is called the adjoint system. In differential geometry, the adjoint system is the 
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differential equations satisfied by the fiber coordinates of an integral curve of the cotangent 
lift of a vector field on Q. See Appendix B .31 for more details. 

Note that there is no initial condition for p = {po,pi, ■ ■ ■ ,Pm), hence HP is not a Cauchy 
problem. 

Comment: So far we have considered a fixed control u £ U. Therefore we have been working 
with a family of Hamiltonian systems on the manifold {T*Q,uj) given by the Hamiltonians 
{H''\ue U}. 

Given u: I ^ U, then we consider the Hamiltonian . The equation of the Hamiltonian 
vector field for the Hamiltonian system (T*Q, ff"*-*^) is 

where dg is the exterior differential on the manifold T*Q. Observe that we have studied the 

system defined by {T*Q,iv, as an autonomous system by fixing the time t. The Hamilto- 

nian vector field obtained is a time-dependent vector field whose integral curves satisfy 

the equation 

= y^^W}(a(t)), tel. (3.10) 

Observe that y^«W> = (X^*)i"(*)>. 

Now we are ready to state Pontryagin's Maximum Principle that provides the necessary 
conditions, which are in general not sufficient, to find solutions of the optimal control problem. 

Theorem 3.14. (Pontryagin's Maximum Principle, PMP) 



3.2 such 



If (7*! 'U*) : I ^ Q xU is a solution of the extended optimal control problem, Statement 
that'y* is absolutely continuous andu* is measurable and bounded, then there exists (a* ,u*) : I 
T*QxU such that: 



1. it is a solution of the Hamiltonian problem, that is, it satisfies Equation (3.10) and the 
initial conditions 7*(a) = (0,Xa) and 7*(6) = X},, if = 112 07*; 

2. 7* = 7rQoa*; 

3. (a) H{a*{t),u*{t)) — sup^g^ i?((T*(t), m) almost everywhere; 

(b) sup ^^jj H (a* {t),u) is constant everywhere; 

(c) a*{t) / G T^,^t)Q for each t G [a,b]; 

(d) (JQ^t) is constant and o'^^t) < 0. 

Comments: 

1. There exists an abuse of notation between a{t) E T*M and a{t) G T^,^^^M. We assume 
that the meaning of a in each situation will be clear from the context. 

2. Condition (2) is immediately satisfied because a* is a covector along 7*. 

3. Conditions (3a) and (36) imply that the Hamiltonian function is constant almost every- 
where for t £ [a,b]. 

4. In item (3a), if [/ is a closed set, then the maximum of the Hamiltonian over the controls 
is considered instead of the the supremum over the controls. But in condition (36) we can 
always consider the maximum, instead of the supremum, because item (3a) guarantees 
that the supremum of the Hamiltonian is reached in the optimal curve. Thus, from now 
on and according to the classical literature, we refer to the assertion (3a) as the condition 
of maximization of the Hamiltonian over the controls. 
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5. Condition (3c) implies that (7^(1) ^ or cr*{t) 7^ G T*,^^^M for each t S [a,b]. Locally 
the condition (3c) states that for each t E [a, b] there exists a coordinate of ^*{t) nonzero, 
{p,od*){t) = a*{t)^0. 

6. From the Hamilton's equations of the system (T*M,uj,H'^^^^), it is concluded that ctq is 
constant along the integral curves of {X'^*)^^^^^\ since po = 0. Hence the first result in 
{3d) is immediate for every integral curve fjo is constant, a* may be 
normalized without loss of generality. Thus it is assumed that either ctq = or cxg = — 1 
because of the second result in {3d) . 

7. Pontryagin's Maximum Principle only guarantees that given a solution of OCP there exists 
a solution of HP. Hence, in principle, both problems are not equivalent. 

Observe that Maximum Principle guarantees the existence of a covector along the optimal 
curve, but it does not say anything about the uniqueness of the covector. Indeed, this covector 
may not be unique. Depending on the covector we associate with the optimal curves, different 
kind of curves can be defined. 

Definition 3.15. A curve (7, u): [a,b] ^ Q x U for OCP is 

1. an extremal if there exist a: [a, b] T*Q such that 7 = vr^g o a and {a, u) satisfies the 
necessary conditions of PMP; 

2. a normal extremal if it is an extremal with = —1 and a is called a normal lift or 
momenta; 

3. an abnormal extremal if it is an extremal with (Tq = and a is called an abnormal 
lift or momenta; 

4. a strictly abnormal extremal if it is not a normal extremal, but it is abnormal; 

5. a strictly normal extremal if it is not a abnormal extremal, but it is normal. 

In [21 [7H] there are some examples of optimal control problems whose solutions are searched 
using Pontryagin's Maximum Principle. 

Observe that if 7 : / ^ Q is an integral curve of a vector field, there always exists a lift of 7 
to a curve a: I ^ T*Q, given an initial condition for the cofibers, which is an integral curve of 
the cotangent lift of the given vector field on Q. Analogously, if the system is given by a vector 
field along the projection n: Q x U ^ Q. 



Therefore, the items 1 and 2 in Theorem 3.14 do not provide any information related with 



the optimality. They only ask for the fulfilment of a final condition in the integral curve. The 
accomplishment of this depends on the accessibility of the problem, see |23| l63] . 

The real contribution of PMP is the third item related with the optimality through the 
maximization of the Hamiltonian, that will be only satisfied if the initial conditions for the fibers 
are chosen suitably. This is the key element of the proof of Pontryagin's Maximum Principle. 
In other words, we can always find a cotangent lift of an integral curve such that conditions 1 
and 2 are satisfied under the assumption of accessibility, but it is not guaranteed the fulfilment 
of conditions in assertion 3 in Theorem 13.141 

If we write the Hamiltonian function for the abnormal and the normal case, the difference is 
that the cost function does not play any role in the Hamiltonian for abnormal extremals. That 
is why it is said the abnormal extremals only depend on the geometry of the control system. 
But to determine the optimality of the abnormal extremals, the cost function is essential. In 
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fact, for the same control system different optimal control problems can be stated depending on 
the cost function, in such a way that the abnormal extremals are minimizers only for some of 
the problems. 

To conclude, the strict abnormality characterizes the abnormal extremals that are not nor- 
mal. An extremal is not normal when there does not exist any covector that satisfies Hamilton's 
equations for normality. Thus it is necessary to know the cost function in order to prove that 
there exists only one kind of lift. 



4 Proof of Pontryagin's Maximum Principle for fixed time and 
fixed endpoints 

To prove Pontryagin's Maximum Principle it is necessary to use analytic results about absolute 
continuity and lower semicontinuity for real functions, and properties of convex cones. For the 
details see Appendix |A] and Appendix |D] and references therein. The reader is referred to §3.3| 
for results on perturbations of a trajectory in a control system. 

In the literature of optimal control, the proof of the Maximum Principle has been discussed 
taking into account varying hypotheses, [21 [HI [TE[ [701 WT\ [72 j. Most authors believe and 
justify that the origin of this Principle is the calculus of variations, see ^76j for instance. 



Proof. (Theorem 3.14' Pontryagin's Maximum Principle, PMP) 

1. As (7*, u*) is a solution of OCP, if r is in [a, b], for every initial condition a-j- in T^,^^.^Q, we 

have a solution of HP, (7*, a) : [a, 6] — > T*Q, satisfying that initial condition. The covector Br 
must be chosen conveniently so that the remaining conditions of the PMP are satisfied. 



According to { 3.3 we construct the tangent perturbation cone Kf, in T^*(^i,^Q that contains all 



tangent vectors associated with perturbations of the trajectory 7* corresponding to variations 



of u*: see Definition 3.11 



Let us consider the vector (— l,0)^.(b) G T;^.(5-)Q, where the zero in bold emphasizes that 
is a vector in T^*(p-^M. The vector (—1,0) has the following properties: 

1. the variation of x^{t) = F{'j*{s),u*{s))ds along (—1,0) is negative; 

2. it is not interior to Kf,. 

Let us prove the second assertion. Take a local chart at 7*(&) and work on the image of 
the local chart, in M™"^^, without changing the notation. 



If (—1, 0);y*(ft) was interior to Kb, by Proposition 3.12 there would exist a positive number 
e, such that for every s G (0,e), there would exist a positive number s' , close to s, and a 
perturbation of the control ^[vr*] such that 

7[7r1(6) = (7°k1(6),7[vr1(6)) = 7*^ + ^'(-1,0). 

For this perturbed trajectory we have 

7°[7r^](6) < -f*\b) and 7[7r*](6) = 7* (6). 

Hence there would be a trajectory, 7[vr''], from 7* (a) to 7* (5) with less cost than 7*. Hence 
7* would not be optimal. In other words, (—1, 0);y.({,) is the direction of decreasing of the 
functional to be minimized in the extended optimal control problem. 
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The second property implies that Ki^ cannot be equal to T^*(^i)-^Q. As (— l,0);y.(fc) is not interior 
to Kh, there exist a separating hyperplane of Kb and (— 1, 0)::y.(f,) by Proposition 



D.15 



that 



is, there exists a nonzero covector determining a separating hyperplane. Let ab G '^^*(b)Q 
nonzero such that kerS";, is a separating hyperplane satisfying 

(?b,(-l,0)) >0, 
{(^b,Vb) < \/vb£ Kb- 

Observe that if af, = G T-^j-j^^Q, ker a^, does not determine a hyperplane, but the whole space 

T^*{b)Q- 

Given the initial condition at S T^^^^-^Q, there exists only one integral curve a* 
such that a*(b) = ab- Hence (a*,u*) is a solution of HP. 

2. Obviously, by construction, 7* = ttq oa* . 

Now we prove that a* , the solution of HP, satisfies the remaining conditions of the PMP. 
(3a) H {a* (t) , u* (t)) = supufzjj H {a* (t) , u) almost everywhere. 

We are going to prove the statement for every Lebesgue time, hence it will be true almost 
everywhere, see Appendix [a| for more details. Suppose that there exists a control u: I ^ U and 
a Lebesgue time ti such that u* does not give the supremum of the Hamiltonian at ti; that is, 

H{a*{ti),u{h)) > H{a*{h),u*{h)). 

As H{p,u) = {p,X{x,u)), 

(a* (t 1 ) , X(7* (t 1 ) , S(t 1 ) ) - X(7* (t 1 ) , n* (ti ) ) ) > 0; 

that is, {a*{ti),v[TTi]) > where ^^[vri] = X{^*{h),u{h)) - X{^*{ti),u*{h)) in Kt, C T^y,)Q 
is the elementary perturbation vector associated with the perturbation data vri = {ti,l,n(ti)} 



by Proposition 3.4 



Let V[7Ti]: [ti,b] TQ be the integral curve of with {ti,j*{ti),v[Tri]) as initial 

condition. For a*, solution of H P, the continuous function (CT*,y[7ri]): [ti,b] ^ M is constant 



everywhere by Proposition B.5 Hence {a*{ti),v[-7Ti]) > implies that (a;,, y[7ri](6)) > 0, which 
is a contradiction with {ab,Vb) < for every Vb S Kb, since V'[vri](6) G Kb- 

Therefore, 

H{a*{t),u*{t)) = sup H{d*{t),u) 
at every Lebesgue time on [a,b], so almost everywhere. 

(3b) sup ^^jj H {a* {t),u) is constant everywhere. 

In fact, because of (3a) we know that the supremum is achieved along the optimal curve, so 
at the end we are going to prove that maxu^u H (X* {t) , u) is constant everywhere. To simplify 
the notation we define the function 

Mod*: I — > M 

t ^ M{a*{t)) =sup^^uH{a*{t),u). 

In order to prove (36), it is enough to see that M{a*{t)) is constant everywhere. 
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First let us see that Aioa* is lower semi continuous on /. See Appendix [A| for details of this 
property. As Ai(a*{t)) is the supremum of the Hamiltonian function with respect to control, 
for every e > 0, there exists a control ujn : I ^ U such that 



(4.11) 



H{a*it),UMit))>Mia*it))-^ 
everywhere. 

For each constant control u £ U, H'^ oa*= H(a* , u): / ^ M is continuous on /. Hence for 
every to G I and e > 0, there exists 6 > such that \ t — to \< 6, we have 

\H^{d*{t))-H^{d*{to))\<'-. 

If 5 = UM{to), then using the continuity of i7" o a* 

M{d*{t)) = snpH{a*it),u) > H{d* {t),UM{to)) > 

> H{d*{to),UM{to)) - I > M{d*{to)) - e. 



The last inequality is true by evaluating Equation (4.11 ) at to- Hence A4 oa* is lower semi con- 



tinuous at every to S /; that is, oa* is lower semicontinuous on /. 

The control u* is bounded, that means Imu* is contained in a compact set D C U. Let us 
define a new function 



Md: T*Q 



As H{[5, • ) : — > M, li 1-^ H{P,u) is continuous by hypothesis and D is compact, for every 
P G T*Q there exists a control W/^ that gives us the maximum of H{(3, u) 



Md{P) = sup H{p, u) = H{P, Wf,). 



(4.12) 



Hence A^£) is well-defined on T*Q. The following sketch explains in a compact way the necessary 
steps to prove that A4 o a* is constant everywhere. In this sketch, the figures in bold refer to 
statements which are going to be proved in the next paragraphs and a.c. stands for absolutely 
continuous and a.e. for almost everywhere. 



7W o CT* is lower semicontinuous on I 



is locally Lipschitz M u £ D 
A^£) is locally Lipschitz on Ima* 
a* is a.c. 



o 5* is a.c. =^ Al/) o a* is continuous 

"^Moid^it)) < Mid*it)), yt£ [a,b] 
^MD{d*{t)) =M{d*{t))a.e. 



A^D o is a.c. 

^ TWd OCT* has zero derivative 



>^ {A. 15) MD{^*{t)) = M{a*{t)) y t e [a,b] 
>^ (A. 16) MDi^ it)) is constant V t G [a, b] 



M{a*{t)) is constant V t G [a, b] 
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1. i?" G C^{T*Q) H" is locally Lipschitz ^uG D. 

The Lipschitzian property applies to functions defined on a metric space. As the property 
we want to prove is local, we define the distance on a local chart as is explained in Appendix 
[a| For every /3 G T*Q, let (Va,(/)) be a local chart centered at (3 such that (j){(3) = and 
(p{Vf^) = B, where B is an open ball centered at G If f5i and P2 are in Vg, define 

d<j}{f3i, /32) = (/)(/32)) where d is the Euclidean distance in 1^2"^+^. 

For every f3 in T*Q, we get an open neighbourhood Vg using the local chart (Vg, (p). As ff" 
is C^{T*Q) and u lies in the compact set D, by the Mean Value Theorem for every /3 in T*Q 
there exists an open neighbourhood Vg such that \H'^{(3i) — i/"(/32)| < Kpd^[(ii^ (52) where Kp 
does not depend on the control u. Thus is locally Lipschitz on T*Q. Moreover, the Lipschitz 
constant and the open neighbourhood Vg do not depend on the control since S is in a compact 
set. 

2. i?" is locally Lipschitz \/u^D^^Ao''^s locally Lipschitz on lm.a*. 
Let (3 be in Ima*, there exists an open convex neighbourhood Vg such that 

|F^(/3i)-i/"(/32)| <K/3d(/3i,/32) 

for every u \n D and (32 in Vg. If wi, W2 are the controls in D maximizing H{(3i,u) and 
H{(32,u), respectively, then 

H{(3uW2) < H{(3i,m), 
H{(32,wi) < H{(32,W2)- 

Moreover, H^^ and H^^ are Lipschitz on Vg since the Lipschitz constant and the neighbourhood 
is independent of the control. Then using the last inequalities 

-Kpd{(3i,h) < - H^'^m < H''^{(3i) - H''^{(32) 

< H^'{(3,)-H^^{p2) < Kpd{^i,p2). 



Observe that by Equation (|412|, H'"^{pi) - H'"^{P2) = Md{(3i) - Md{I32)- Hence 

\MD{(3i)-MD{(32)\<Kpd{(3i,(32), ^ (3i, (32 e Vp; (4.13) 

that is, A4d is locally Lipschitz on Ima*. As a* is absolutely continuous, Ima* is compact. 
Thus we may choose a Lipschitz constant independent of the point p. Hence 

\Md{Pi) - Md{P2)\ < Kd{(3i,p2), y(3i,(32eV^. 

3. is locally Lipschitz on Ima* and a* is absolutely continuous =^ Ma '■ / ^Mis 
absolutely continuous ^ M.d '■ / ^Mis continuous. 



For every i G /, let us consider the neighbourhood V^*(^f^ where Equation (4.13) is satisfied. 
As Ima* is a compact set, 

• there exists a finite open subcovering Vg:*(t^), . . . , of {V5=*(t) ! t ^ I}, and 

• there exists a Lebesgue number I of the subcovering; that is, for every two points in an 
open ball of diameter I there exists an open set of the finite subcovering containing both 
points. 

For the Lebesgue number /, by the uniform continuity of a* , there exists a (5/ > such that for 
each ti, t2 in I with \t2 — ti| < 61, then d(a*{t2),a*{ti)) < I. Thus there exists an open set of 
the finite subcovering containing and a*(t2)- 

On the other hand, taken e > the absolutely continuity of a* determines a 6^ > 0- 
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To prove the absolute continuity of TW/j o a*, take 5 = min{5/,5e}- Then, for every finite 
number of nonoverlapping subintervals {ti^,ti^) of / such that XlILi l^«2 ~ ^nl < 

n n 

Y.\MD{^*{k,)) - MD{^*{k,))\ <J2Kd{a*{ti,),a*{U,)) < Ke. 

i=l 1=1 

In the first step we use that 6 < 6i to guarantee that a*{ti2) and a*{tij^) are contained in the 
same open set of the finite subcovering of Im a* . That ahows us to use the property of being 
locally Lipschitzian. Secondly, we use that 5 < 6e to apply the absolute continuity of a* . 

As Aif) oa* is absolutely continuous on I, oa* is continuous on /. 

4. MD{^*{t)) <M{a*{t)) everywhere. 

Observe that 

MD{^*{t)) = sup H{a*{t),u) < sup H{a*{t),u) =M{a*{t)), 

for each t £ I. 



5. M{d*{t)) = 7Wz)(?*(t)) almost everywhere. 

For each t £ I there exists a control w^t) maximizing H(a*{t),u) over the controls in D 
because of condition (3a), 

MD{y{t)) = sup H{a*{t),u) = H{a*{t),w{t)). 

As u*{t) e D for each t £ I, 

sup H{a*{t),u) = sup H {a* {t),u) =M{a*{t)) = H{a*{t),u*{t)) 

almost everywhere by (3a). Thus Ai{a*{t)) = (t)) a.e.. 



6. Applying Proposition A. 7 we have A4D(5*(i)) = M(jJ*{t)) everywhere on /, because 
A4d°^* is continuous on /, Moa* is lower semicontinuous, Mr)(a* (t)) < M(a*(t)) everywhere 
and A^D(5*(t)) = M.{a*{t)) almost everywhere. 

7. oa* has zero derivative. 

As Air) ° ^* is absolutely continuous on I, by Corollary |A.4 it has a derivative almost 
everywhere. As the intersection of two sets of full measure is not empty, see Appendix [Aj there 
exists a to £ I such that Md ° o"* is derivable at to and MD(d'*{tQ)) = H {a* {to) , u* (to)) . For 
each t ^ to, by the definition of A^d, we have 

MD{^*{t)) = sup H{a*{t),u) > H{a*{t),u*{to)) 

because u*{to) £ D. Thus M d{^* it)) - M d{^* {to)) > H {d* {t) , u* {to)) - H{d*{to),u*{to)). 
If t - to > 0, 

MD{a*{t))-MD{a*{to)) ^ H{a*{t),u* {to)) - H{a*{to),u*{to)) 



dt 



t-to 



^. MD{d*{t)) - MD{a*{to)) ^ H-*(''>)^^*^t)) - F-*(*o)(a*(to)) 
hm > lim 



t-to 

Let us compute the right derivative oi Md oa* at to 
d{MD°a*) 

= iiiii 

t=t+ t^t+ t - to 



t-to 



^3*(t0) 
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since ^(lI4";;*»'>). = (di/«*('o))^.(^^^. 
Similarly, if t — to < 0, 



d{M D o 5* 



dt 



< 0. 



t=t~ 



Hence the derivative of A^/j o ct* is zero almost everywhere. 



8. Applying Theorem |A.5[ M-d is constant everywhere, because M-d is absolutely 
continuous. 



9. As Aioi^* {t)) and Ai{a*(t)) coincide everywhere, Ai oa* is constant everywhere on /. 



(3c) d*{t) / G T|.(^)Q for each t G [a, b]. 

Let us suppose that there exists r G [a, 6] such that 5^*(t) = G T^t^^^^Q. As a* is a 

generalized integral curve of (X-^*)^"*^, a linear vector field over X, we have a*(t) = for each 
t G [a, b]. As there exists at least a time such that ?*(t) 7^ 0, we arrive at a contradiction. Hence 
a*{t) ^ for each t G [a, 6]. 

(3d) (jQ(t) is constant, (jQ(t) < 0. 

From the equations satisfied by the generalized integral curves we have po is 

constant. It was seen that {Bh, (—1,0)) > is equivalent to {po o d*){b) = o"o(6) < 0. Hence 

(70 < for each t £ [a,b]. □ 

Comment: As is determined up to multiply by a positive real number, we may assume that 
G {-1,0}. 

The way in which perturbations have been used in this proof gives some clue concerning the 
fact that the tangent perturbation cone is understood as an approximation of the reachable set 
defined in Appendix [Cj A precise meaning of this approximation is explained in Appendix [Cj 

The covector in the proof has been chosen such that 

(at, {-1,0)) >0, 
{'(^b,Vb) < G Kb. 

In the abnormal case do = and the first inequality is satisfied with equality. Thus the covector 
is contained in the separating hyperplane. It would be interesting to determine geometrically 
what else must happen in order to have abnormal minimizers. 



5 Pontryagin's Maximum Principle for nonfixed time and non- 
fixed endpoints 

Now, that Pontryagin's Maximum Principle has been proved for time and endpoints fixed, let 
us state the different problems related to Pontryagin's Maximum Principle with nonfixed time 
and nonfixed endpoints. 

5.1 Statement of the optimal control problem with time and endpoints non- 
fixed 



We consider the elements Q, U, X, F, S and tt2 with the same properties as in ^ ^3.1 Let Sa 
and 5*/ be submanifolds of Q. 
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Statement 5.1. (Free Optimal Control Problem, FOCP) Given the elements Q, U, X, 
F, and the disjoint suhmanifolds of Q, Sa and Sj, consider the following problem. 

Find 6 G M and (7*, n*) : [a,b] ^ Q x U such that 

(1) endpoint conditions: 7*(a) G Sa, 7*(&) G Sf, 

(2) 7* is an integral curve of X^^*^ : 7* = X^^*^ o (7*, id), and 

(3) minimal condition: S[j*,u*] = J^^ F{'~f*(t),u*(t))dt is minimum over all curves (7,ti) 
satisfying (1) and (2). 

The tuple (Q, U, X, F, Sa,Sf) denotes the free optimal control problem. 
Statement 5.2. (Extended Free Optimal Control P roble m, FOCPj Given the FOCP, 



(Q, U, X, F, Sa, Sf), and the elements Q and X defined in { 3.2, consider the following problem. 
Find 6 € M and (7*, u*): [a,b] Q x U , with 7* = 7r2 o 7*, such that 

(1) endpoint conditions: 7*(a) G {0} x Sa, 7*(&) G Sf, 

(2) 7* is an integral curve of X^^*\- 7* = X^^*^ o (7*, id), and 

(3) minimal condition: 'J*^\b) is minimum over all curves (7, u) satisfying (1) and (2). 
The tuple {Q, U, X, Sa, Sf) denotes the extended free optimal control problem. 

5.2 Perturbation of the time and the endpoints 

In this case of nonfixed time and nonfixed endpoint optimal control problems, we not only modify 



the control as explained in ^3.3 but also modify the final time and the endpoint conditions. As 
was mentioned in §3.3[ the following constructions obtained from perturbing the final time and 
the endpoint conditions are also general for any vector field depending on parameters. 

5.2.1 Time perturbation vectors and associated cones 

We study how to perturb the interval of definition of the control taking advantage of the fact 
that the final time is another unknown for the free optimal control problems. 

Let X be a vector field on M along the projection tt: M x U M, / C M be a closed 
interval and (7, u) : / = [a, 6] ^ M x [/ a curve such that 7 is an integral curve of XW. 

Let TT± = {t,It-,St,Ut-}, where r is a Lebesgue time in (a, 6) for X o (7,n), Ir G U {0}, 
St G M, G U. For every s G small enough such that a < t — {1^- — 5t)s, consider 
u[-K^] : [a,b + drs] U defined by 

u{t), t £ [a,T — {It — 6t).s], 

t £ { T — {Ir — 6t)s, T + 6tS ] , 

u{t), t £ {t + 6ts, b + 6ts ] , 




t £ [a,T — {Ir — (5r)s], 

t G ( T — (/r — 5t)s, T + 5tS ] 

u{t — 6ts), t G ( t + 5ts, b + 6ts ] , 



if 6t > 0. 
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Definition 5.3. The function ti[vrj_] is called a perturbation of u specified by the data 

7r± = {t, lr,ST, Ur}. 



Associated to u[tt^] we consider the mapping 7['7r^] : [a, b+5Ts] M, the generahzed integral 
curve of X^"''^±lj' with initial condition (0,7(0)). 



Given e > 0, define 



: [r, b] X [0, e] ^ M 

{t,s) I — > (/7^±(t, s) =7[7r4](t + 5ts) 

For every t G [r, 6], ip^^ : [0, e] ^ M is given by ipi^{s) = ipn±{t, s). 



As explained in ^3.3 the control u[7r^] depends continuously on the parameters s and vr-t = 
{r, (5r, n,-}. Hence the curve 93^^ depends continuously on s and tt± = {T,lr,ST,Ur}, then it 
converges uniformly to 7 as s tends to 0. See |251I31] for more details of the differential equations 
depending continuously on parameters. 



Let us prove that the curve 99^^ has a tangent vector at s = 0; cf. Proposition 3.4 



Proposition 5.4. Let t be a Lebesgue time. If u[Tr^] is the perturbation of the control u 
specified by the data 7r± = {T,lr,ST,Ur} such that r + s5t is a Lebesgue time, then the curve 
: [0,e\ ^ M is differentiable at s = 0. Its tangent vector is 

X{j{t),u{t)) 6t + [X{^{T),Ur) - X{j{t),u{t))] Ir. 

Proof. As in the proof of Proposition |3.4[ we compute the limit 



A = lim = lim 

s^O S s^O S 

As 7 is an absolutely continuous integral curve of , 7(f) = X{-f{t),u{t)) at every Lebesgue 
time. Then by integration 

y(r)-y(o)= r f\^{t)Mt))dt 

J a 

and similarly for 7[vr^] and n[7r^]. Observe that 7[7rj.](f) = 7(f) and u[7r^](f) = u{t) for t G 
[a, T — [It — 6t)s\. 

Here, we should consider three different possibilities 

• if < (5r < /t, then t — {Ir — 5t)s < t < t + 5ts; 

• if (5r < 0, then t — {Ir — 5t)s < t + 6ts < r; 

• if < < 5r, then t < t — {It — 6t)s < r + 6ts. 

We prove the proposition for the first case and the other cases follow analogously. 
^ ^ r (7[vr|](t), n[vr|](t))dt - £ f\j{t),u{t))dt 

s^O S 

= lim ^ ^ . 

S 
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We need r + 6ts to be a Lebesgue time in order to use Equation (A. 17). 
A = lim 



f (7ki](T + 5Ts),Ur)lrS " f , ^(t)) - 6t)s + o{s) 



= lim r(7ki](r + 5Ts),Ur)lr - f{l{r),u{T)){lr - 5t). 
As /* is continuous on M, we have 

A = [filiT),Ur)-fh{T),u{T))]lr + f{-/{T),u{T))6T 

= L([X(7(r),n(T)) 5r+(X(7(r),n.)-X(7(r),tx(r))) /.])(x^). 
Definition 5.5. The tangent vector 

v[Tr±]= X{j{t),u{t)) 6T+[X{j{T),Ur)-X{^{T),u{T))] 1^ 

is the perturbation vector associated to the perturbation data 7r± = {r, 5r, n,-}. 



□ 



If we disturb the control r times at r different Lebesgue times as in §3.3.1 and also the domain 
of the curve (7, u) as just described, that is, n = {vri, . . . , vr^., tt±}, with a < ti < . . . < tr < t < b, 
then 7['7r''] is the generalized integral curve of with initial condition (0,7(0)). Consider 

the curve 93^ : [0, e] ^ M for t G [r, b] given by ipi{s) = 7[vr'^](t + 5ts). 

Corollary 5.6. The vector tangent to the curve (p\^ : [0, e] ^ M at s = is X(7(t), u{t)) 5t + 

y[7ri](t) + . . . + y[7r„](t), where V[Ki] : [ti, b] TM is the generalized integral curve of (X^)^"^ 
with initial condition (ti, {^{ti),v[TTi])). 



This corollary may be proved taking into account Propositions 3.6 and 5.4 Corollary |3.9| 
and Appendix [B| 

Now, at a Lebesgue time t £ (o, b), we construct a new cone that contains the perturbation 



vectors in Definition 3.11 and ±X{j{t),u{t)). 



Definition 5.7. The time perturbation cone at every Lebesgue time t is the smallest 
closed cone in the tangent space at 7(t) containing Kt and ibX(7(t), u(t)), 



= conv 



{±aX{j{t),u{t))\a£R}[J 



V 



u 



{u} 

{t,r) 



Vr 



. a<T<t 
\t is a Lebesgue time 



where Vr denotes the set of elementary perturbation vectors at r, see Definition 3.11 

Enlarging the cone Kr to allows us to introduce time variations. 
Proposition 5.8. 7/^2 is a Lebesgue time greater than ti, then 



Proof. We have 



Kf^ = conv 



/ / 

{±aX(7(ti),ii(ti))|aGM}|J 



V 



U 



a<T<t\ 
T is a Lebesgue time 



J 
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Just for simplicity we use to denote 

/ / 

{±aX{^{ti),u{ti))\aeR}\J 



conv 



V 



U 



a<T<ti 
T is a Lebesgue time 



Hi::,} V. 



1. The set being convex, if v is interior to Kl^, then v is interior to by Proposition 



D.5 item (d). Hence, by Proposition D.4 



V = StiX{^{ti),u{h)) + ^ liV[7ri]{ti), 



i=l 

where every T^[7ri](ti) is the transported perturbation vector f [vrj] of class I from ti to ti 
by the flow of X^^\ By definition of the cone and the linearity of the flow, (^^'(tl'^ti)) ^ 

is in Kf^^ since {^ftl'^ti)) ("'^(tC^i)' ~ -^^(7(^2), ^(i2))5 because both sides of the 
equality are the unique solutions of the variational equation along 7 associated with X^"^^ 



with initial condition {ti,X{'^{ti)^u{ti)). See Appendix B.2 for more details. 



2. If w is in the boundary of i^^, then there exists a sequence of vectors (t'j)jeN in the interior 



of Kt^ such that 

Due to the continuity of the flow 

lim ( <I> 



lim Vj = V. 

j->oo 



All the elements of the convergent sequence are in the closed cone -fC^, hence the limit 

If the interior of K^^ is empty, we consider the relative topology and the reasoning follows 
as before. See Appendix [P] for details. 

□ 

For the time perturbation cone and the corresponding perturbation vectors, it can be 
proved properties analogous to the ones stated in Propositions 3^, 3^ |3.7 and 3^ 



Proposition 5.9. Let t £ {a,b) be a Lebesgue time. If v is a nonzero vector interior to , 
then there exists e > such that for every s S (0, e) there are s' > and a perturbation of the 
control u[7r^] such that 7[7r^](t + s5t) = 7(t) + s'v. 



Proof. The proof follows the same line as the proof of Proposition 3.12 but now the tangent 
space to M at 7(t + sSt) is also identified with through the local chart of M at j{t). 



We use the same functions as in the proof of Proposition 3.12 but keeping in mind that 



r(s,r) =7[<o](*) is replaced by r(s,r) = j[iTl ]{t + s6t). 



□ 



29 



5.2.2 Perturbing the endpoint conditions 



Now we consider that the endpoint conditions for the integral curves of Xi"} varies on submani- 
folds of M. Let Sa be a submanifold of M and 7(a) in Sa] consider the integral curve 7: I ^ M 
of X^^^ with initial condition (a, 7(a)). 

We consider the curve 7[vr^] obtained from a time perturbation of the control u associated 
with a vector in the time perturbation cone. The initial condition is disturbed along a curve 
S: [0,e] Sa with initial tangent vector Va in T^(^a)Sa and (5(0) = 7(a). Taking into account 



Appendix B.2.1 ^3.3.1 and considering that T^(^a)Sa and an open set at 5{a) are identified with 
M™, the integral curve 7^(<j)[7r^]: I ^ M of X^"['^±1j' with initial condition (a, can be 
written as 

l5(s)[T^±\{t) = lit) + S (^J,!?)^ + Sv[7r±]{t) + 0{S). 

We define a cone that includes the time perturbation vectors, the elementary perturbation 
vectors and the vectors coming from changing the initial condition on Sa along different curves 
5: [0,e] — > Sa through 7(a) and contained in Sa- 

Definition 5.10. Let t be a Lehesgue time. The cone fCt is the smallest closed and convex cone 
containing the time perturbation cone at time t and the transported of the tangent space to Sa 
from a to t through the flow of X^'^^ . 

Kt = conviKt [Ji^%'UT,ia)Sa)) 

Proposition 5.11. Lett be a Lebesgue time in (a, b) and S <Z M be a submanifold with boundary. 
Suppose that ^{t) is on the boundary of S. Let T be the half-plane tangent to S at 7(t). If K,t 
and T are not separated, then there exists a perturbation of the control n[7r^] and Xa £ Sa such 
that the integral curve ^xa[''^±\ of X^^^'^±^^ with initial condition {a,Xa) meets S at a point in the 
relative interior of S. 



Proof. As fCt and T are not separated, by Proposition D.15 there no exists any hyperplane 



containing both and there is a vector v in the relative interior of both Kit and T. By Corollary 



D.16 if K-t and T are not separated, 

r^(t)M = ICt-T. 

See Appendix [P] for the notation and properties. If V is an open set of a local chart at 7(t), 
we identify V with and also the tangent space at 7(t), T^(t'jM, in the same sense defined 
for Equation (3.4). Let us consider an orthonormal basis in T^(£^M, {ei, . . . ,em}- If we take 
eo = -(ei+. . .+6™,), the vector € T^(t)M is expressed as an affine combination of 60, 61, • • • , 6m- 

1 1 

= — eo + . . . H TT^m- 

m + 1 m + 1 

Each w in T^(t-^M is written uniquely as 

w = a^ei + . . . + a^'Cm 
and as an affine combination of cQ) ^i, . . • , em'- 

m m ^ ^ y^m 

w = 'S^ b\w)ei = reo + 'S^(r + a'-)ei with r= — , 

^-^ m + 1 

i=0 1=0 

Hence, we define the continuous mapping 



w 1 — > (6°H,6iH,...,6"H). 
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As 6*(0) > for every i = 0, . . . ,m, there exists an open ball -6(0, r) centered at with radius 
r such that for every w S B(0,r), U{w) > for i = 0, . . . ,m. Now we consider the restriction 
of g to the closed ball 5(0, r), Q^io^- B{0,r) [0, 1]™+^ Choose vectors ef G /C* and ef G T 
such that 



As V lies in the relative interior of both convex sets, ej = {ef + v) — {ej + v) = ef — ef . Then 
ef and ef can be chosen in the relative interior of /C and T, respectively, because v is in the 
relative interior of both. For any w £ B{0,r), 

m m 

Y,h\w)e, = Y,b\w) {ef-ej). 



w 

i=0 i=0 



Then we can define 



Fi:B{0,r) Kt 
F2 : 5(0, r) T 



and let us consider the mapping 



G:RxB{0,r) — > R"^ 

{s,w) ^ (7K(^)](i)-7[vr^,(»)](0)A, 

where 7[7r|;.^^^^] is the perturbation curve associated to 7r|,j(^) and 'y['^p^(^uj)]i't) = 7(t) + 5^2(11;) 
is the straight line through 7(i) with tangent vector F2{w). As the perturbation vectors are in 
the relative interior of the convex cones, we use the linear approximation in (3.4) in such a way 
that G{s, w) = Fi{w) - F2{w) + o(l). Hence 

lim G(s, w) = Fi{w) — F2{w) = w, 
s— >o 

Hence, for any positive number e, there exists sq > such that if s < sq then ||G(s, if;) — < e. 
Take e < r, then 

||G(s, w) — ?i;||<e<r=||tt;|| 



for every w in the boundary of 5(0, r). Thus the map Gg'- B{0,r) M"*, Gs{w) = G{s,w), 
satisfies the hypotheses of Corollary |E.2 for the point in 5(0, r). Hence, the point is in the 
image of 5(0, r) through Gs and there exists w such that Gs{w) = 0; that is. 

Therefore, there exists a perturbation of the control such that the associated trajectory meets 
S in an interior point since F2{w) lies in the relative interior of T. □ 



5.3 Pontryagin's Maximum Principle with time and endpoints nonfixed 



Bearing in mind the symplectic formalism introduced in { 3.4 we define the corresponding Hamil- 
tonian Problem when the time and the endpoints are nonfixed. 

Statement 5.12. (Free Hamiltonian Problem, FHP ) Given the FOGP, (Q, U, X, F, Sa, Sj), 
and the equivalent FOGP, {Q, U,X, Sa, Sf), consider the following problem. 

Find 6 G M and (a, u) : [a, b] T*Q x U, with 7 = vr^ o a and 7 = '/r2 o 7, such that 
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(1) 7(a) e {0} X Sa, -fib) G Sf, and 

(2) j = (X^*)Wo(a,id). 

The tuple {T*Q, U, X'^ , Sa, Sf) denotes the free Hamiltonian problem. 
Comments: 

1. The minimum of the interval of definition of the curves is o, but the maximum is not fixed. 

2. The curves 7, 7 and a are assumed to be absolutely continuous. So they are generalized 
integral curves of X^"^, X^^^ and (X^*)^"^, respectively, in the sense defined in ^ 

Now, we are ready to state the Free Pontryagin's Maximum Principle that provides the 
necessary conditions, but in general not sufficient, for finding solutions of the free optimal 
control problem. 

Theorem 5.13. (Free Pontryagin's Maximum Principle, FPMP) 

If (7 * ,u*): [a, 6] — > Q X [/ is a solution of the extended free optimal control problem, Statement 



5.2. then there exists (a*,u*): [a,b] T*Q x U such that: 

1. it is a solution of the associated free Hamiltonian problem; 

2. 7* =7rQod*; 

3. (a) H{d*{t),u*{t)) — supygf/ i?(fT*(t) , u) almost everywhere; 

(b) sup ^^jj H (a* {t),u) = everywhere; 

(c) a*{t) / G T^,^t)Q for each t G [0,6]; 

(d) cTo(t) is constant, crQ{t) < 0; 

(e) transversality conditions:a* (a) G ann T^. 'S'a and cr*{b) G ann T^. (fe) S"/ . 

Observe that once we have the optimal solution of the FOCP, the final time and the endpoint 



conditions are known and fixed. We would like to apply just Theorem 3.14 in order to prove 



Theorem 5.13 However, this is not possible because the freedom to chose the final time and 



the endpoint conditions, only restricted to submanifolds, in Statement 5.2 is used in the proof 
to consider variations of the optimal curve that are slightly different from the variations used in 
the case of fixed time, see §3.3| and §5.2| to compare them. 

Apart from the transversality conditions, the main difference between FPMP and PMP is the 
fact that the domain of the curves in the optimal control problems is unknown. That introduces 
a new necessary condition: the supremum of the Hamiltonian must be zero, not just constant. 
Then, from (3a) and (36) it may be concluded that the Hamiltonian is zero almost everywhere. 
For instance, in the time optimal problems the Hamiltonian along extremals must be zero. 

There are different statements of Pontryagin's Maximum Principle. In §3.4| we have con- 
sidered the statement of PMP for a fixed-time problem without transversality conditions to 
simplify the proof, although it may be stated the PMP for the fixed-time problem with variable 
endpoints where the transversality conditions appear. There also exists the PMP for the free 
time problem with the degenerate case that the submanifolds are only a point, then the Theorem 
is the following one. 

Theorem 5.14. (Free Pontryagin's Maximum Principle without variable endpoints) 

If (7 * ,u*): [a,b] ^ Q X U is a solution of the extended free optimal control problem. Statement 



5.2 with Sa = {xa} and Sf = {xf}, then there exists {a*,u*): [a,b] T*Q x U such that: 
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1. it is a solution of the associated free Hamiltonian problem; 

2. 7* 



3. (a) H{a*{t),u*{t)) — sup^gjj -ff ((T*(t), u) almost everywhere; 

(b) sup ^^jj H {a* {t),u) = everywhere; 

(c) a* it) / G T1,^^^^Q for each t G [o, 6]; 

(d) al{t) is constant, (TQ{t) < 0; 



The only difference with Theorem 5.13 is that the transversality conditions do not appear. 



6 Proof of Pontryagin's Maximum Principle for nonfixed time 
and nonfixed endpoints 



In the proof of Theorem 5.13 we use notions about perturbations of the trajectories of a system 



introduced in ^5.2, but they are slightly different from the perturbations in ^3.3 used to prove 
Theorem 13.141 



Proof. (Theorem 5.13' Free Pontryagin's Maximum Principle, FPMP) 



Given a solution of the FOCP, we only need an appropriate initial condition in the fibers of 
TT^: T*Q ^ Q to find a solution of the FHP, because this initial condition is not given in the 



hypotheses of the Free Pontryagin's Maximum Principle. It is not possible to use Theorem 3.14 
directly because the perturbation cones are not the same. Indeed, we need to consider changes in 
the interval of definition of the curves. These changes imply the inclusion of zizX {j* {ti) , u* (ti)) 
in the perturbation cone at time ti. All the times considered in this proof are Lebesgue times 
for the vector field giving the optimal curve. 



By Proposition 5.8 for t2 > ii, 



(t2,tl) 



Let us consider the limit cone as follows 



U 



a<T<b 
r is a Lebesgue time 



Observe that it is a closed cone and it is convex because it is the union of an increasing family 
of convex cones. Let us show that (— l,0);y«(fc) is not interior to K^. Indeed, suppose that 
(— l,0);^»(fc) is interior to the limit cone, then it will be interior to 



u 



a<T<b 
T is a Lebesgue time 



by Proposition D.5 item (d). As we have an increasing family of cones, there exists a time r 
such that (— 1, 0);y.(;,) is interior to Let us see that this is not possible. 

If 



-l,0);^*(b) is interior to then, by Proposition 



5.9 



there exists e > such 



that, for every s £ (0, e), there exist s' > and a perturbation of the control ^^[vr^g] such that 
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7°KJ(6 + s6t) < 7* (6) and 7KJ(6 + s6t) = j*{b). 



Hence 

That is, the trajectory 7['/r^Q] arrives at the same endpoint as 7* but with less cost. Then 7* 

I. Thus (— l,0);y.(b) is not interior to K^. 

there exists a covector 



cannot be optimal as assumed. Thus (— l,0);y.(b) is not interior to K^. 
As {—l,0)j*(^i,-j is not in the interior of K^, by Proposition 



D.15 



CTfe E T^^*{h)Q such that 

(afe,(-l,0)) > 0, 

The initial condition for the covector must not only satisfy the previous inequalities, but also 
the transversality conditions. In order to prove this, it is necessary to have the separability of 
two new cones. 

(3e) Hence, the initial condition in the fibers of T*Q may be chosen satisfying the transver- 
sality conditions. We consider the manifold with boundary given by 

Mf = {{x^,x) I a; G 5/, x° < 7*°(6)}. 

The set of tangent vectors to Mj at 7* (6) is the convex set whose generators are (— l,0)^.(t) 
and Tf = {0} x T^*(i,)Sf. 

Given r G [a, 6], consider the following closed convex sets 



Kr = COnv(i^± y^i^fr a) U^a)), where Ta = {0} X T^,(,)5a, 



Jr = conv((-l,0)^*(,) \J{^f,%*')^HTf)), where Tj = {0} x T^,^t)Sf, 

and the manifold Mj- obtained transporting Mf from 6 to r using the flow of Observe 
that J'r is the closure of the set of tangent vectors to Mj- at the point 7*(t). We are going to 
show that the cones /Cf, and are separated, using Proposition |5.11 



Observe that is a half-plane tangent to Mf and 7* (6) is on the boundary of Mf by 



construction. Hence, if /Cf, and ^b were not separated, by Proposition 5.11| there would exist a 
perturbation of the control ti[vr^Q] and Xa G Sa such that the integral curve 7xa[7r^o] with initial 
condition (a, Xa) meets Mf at a point in the relative interior of Mf. Hence we have found a 
trajectory with less cost than the optimal one because of the definition of Mf and this is not 
possib le because of the optimality of 7*. Thus JCb and J'b are separated. So, by Proposition 



D.15 



there exists a covector ab G T2^*Qj)Q such that 



{db.Vb) < V^JbG/Cft, (6.14) 
{'5b. Wb) > yivbGJb. (6.15) 

This covector separates the vector {—l,0)^*(^b) ^ -^fo ^-nd the cone C /C^. Let a* be the 
integral curve of (X^*)'f"*J' with initial condition ab G T^*(b)Q ^• 

As Tf is contained in Jb. we have {ab,v) > for every v G Tf. As Tf is a vector space, if 
V £ Tf, then —v G Tf. Hence, we have 

{ab,v) = for every v G Tf. 

That is, 

{ab, (0,f)) = for every v G T^*(b)Sf. 
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This is equivalent to {crb,v) = for every v E T^*^i,-^Sf] that is, ab = <7*(6) is in the annihilator 
of T^,(b)Sf as wanted. 

For every Wb E if VF: / —^_TQ is the integral curve of {X'^)^^*^ with initial condition Wb 

the pairing continuo us na tural function (a* , W) : / — > M is 



B.5 



at time 6, then by Proposition 

constant everywhere and (?*(a), W{a)) > by Equation (6.15). As {^fb% ^)*^{'^b) = Ja by the 
continuity and the linearity of the flow, the transversality condition at a is proved analogously 
as the transversality condition at h proved above. 



Since (7*, u*) is a solution of the FOCP, it is also a solution of OCP with time and endpoints 
fixed and given by the curve. Henc e, we can apply Pontryagin's Maximuni_Principle for time 
and endpoints fixed. Theorem 



3.14 



If the curve (7*,ii*) is a solution of OCP with / = [a, h] 
and endpoints 7*(a) and 7*(6), (a*,u*): [a, 6] — > T*Q x [/ is a solution of the HP, such that 
7* = tTq o a* , and moreover a* satisfies that 

(3a) H id* it), u* it)) — sup„gf/ i?(fT*(t) , u) almost everywhere. 

(3b) swp^^u Hid* {t),u) is constant everywhere. 

(3c) d*{t) / G T|,(j)Q for every t G [a, b]. 

(3d) (TQ{t) is constant, (TQ(t) < 0. 

Observe that it only remains to prove (3b) of the Free Pontryagin's Maximum Principle, 



since (3a), (3c) and (3d) are the same in both Theorems 3.14, 5.13 



(3b) Due to (3a) we already know that the supremum of the Hamiltonian is constant every- 
where along (a*, t(*). Now, let us prove that the supremum can be taken to be zero everywhere. 

Take % = ±X (j* {b) , u* {b)) G K^, let V : I ^ TQ be the integral curve of (X^)^"*} with 
initial condition (6, 7(6), Vf,), then the continuous function (a*, V): / ^ M is constant everywhere 
by Proposition |B.5[ Thus, 

(a*(t), V{t)) = {a*{t), ±X{^*{t),u*{t))) < for every t G I 



by Equation (6.14), and this implies that 

{d*{t),x{r{t),u*m 



0. 



As {a* (t) , X {■J* (t) , u* (t))) = H (a* {t) , u* {t)) , the Hamiltonian function is zero everywhere and 
the supremum of the Hamiltonian function is zero everywhere by Theorem 3.14 □ 



Observe that the initial condition for the covector in this proof has been chosen such that the 
tangent spaces to the initial and final submanifolds are contained in the separating hyperplane 
defined by the covector. In this statement of the Maximum Principle the initial condition for 



the covector must satisfy more conditions than in Theorem 3.14 (namely the transversality 
conditions). 



Appendices 

This last part of the report is mainly devoted to state and prove some of the results used in the 
proof of the Maximum Principle and to give more understanding to some key points. 
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A Results on real functions 



In this Appendix we focus on some necessary technicalities for the proof of Pontryagin's Maxi- 
mum Principle. These are related with results from analysis and the notion of a Lebesgue point 
for a real function. In this paper the notion of a Lebesgue point is applied to vector fields. For 
more details about all this, see [Ml EH 175] . 

Definition A.l. Let {X,dx) and (Yjdy) be metric spaces. A function f : X ^ Y is Lipschitz 

if there exists K gM such that (iy(/(xi), /(X2)) < K dx{xi,X2) for all xi,X2 € X. 

A function f : X ^ Y is locally Lipschitz if, for every x G X there exists an open neigh- 
bourhood V of X and K G such that dy(/(xi), /(a;2)) < K dx{xi,X2) for all xi and X2 in 
V. 

If M is a differentiable manifold, 5 is a Riemannian metric on M and : M x M — > M is 
the induced distance; then (M, dg) is a metric space where the notion of Lipschitz on M can be 
defined. A real-valued function F: M — > M is locally Lipschitz if, for every p £ M we take the 
local chart (V, (p) such that (j){p) = 0, (piV) = -6(0, r) is the open ball centered at the origin with 
radius r > in the standard Euclidean space, and F o (p~^ : B(0,r) ^ M is Lipschitz. That is, 
there exists K £ M"^ with 

\F{pi) - F{p2)\ = \{F o 4>-^){(t>{p,)) - {F o 4>-^){<p{p2))\ < Kd{^{pi),<f>{p2)), ypi,P2 e V. 

Hence, given the local chart {V,(j)), we define a distance d^: V x V ^ on V, d^{pi,p2) = 
di<P{Pi) J 4'iP2)) ■ Consequently, {V,(j)) is a metric space with the topology induced by the open 
set V in M. This distance is equivalent to the distance induced by the Riemannian metric on 
y. Observe that the notion of locally Lipschitz for functions on manifolds depends on the local 
chart, but functions are always locally Lipschitz. 

Definition A. 2. A function f: [a,b] ^ M is uniformly continuous on [a,b] if, for every 
e > 0, there exists 6 > such that for any t,s £ [a, b] with \t — s\ < 6, we have \f{t) — f{s)\ < e. 

Definition A. 3. A function /: [a,6] — > M is absolutely continuous on [a, 6] if, for every 
e > 0, there exists 5 > such that for every finite number of nonoverlapping subintervals {ai, hi) 
of [a, b] with Yli=i l^i - ^il < ^, we have Yh^^ - /(aj)| < e. 

We consider an interval / = [a, b] in M with the usual Lebesgue measure. A statement is said 
to be satisfied almost everywhere if it is fulfilled in I except on a zero measure set. A measurable 
subset ^ C / is said of full measure if / — ^ has measure zero. Recall that \l A,B C / and 
L — A, L — B have measure zero, then Ar\ B \s not empty. 

Results in [66], pp. 96, 100, 105 allow one to prove the following result. 

Proposition A. 4. If f is absolutely continuous, then f has a derivative almost everywhere. 

Theorem A. 5. [166], pp.105 and 1741, pp.836] If f is absolutely continuous and f'(t) = almost 
everywhere on [a, 6], then f is a constant function. 

Definition A. 6. A real-valued function f on a metric space {X, d) is called lower semicon- 

tinuous at xq G X if, for every e > 0, there exists 5{e,XQ) > such that f(x) > /(xq) — e 
whenever d{x,xo) < 5(e,to)- 

U f lower semicontinuous at every point of {X, d), it is said to be lower semicontinuous 
on (X, d). 

The following result is stated by Pontryagin et al. in [M], page 102, but it is neither proved 
nor stated as a proposition. We believe it is appropriate to write it with more detail because it 
is used in Q 
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Proposition A. 7. Let f and g be real functions, f,g: [a,b] ^ M. /// is continuous, g is lower 
semicontinuous, f < g and f = g almost everywhere then f = g everywhere. 

Proof. Let to ^ [o;^]- As g is lower semicontinuous on [a,b], for every e > there exists 
S{e, to) = 6 > such that 

9{t) > g{to) - e 

whenever \t — to\ < S{e,to). 

Since / and g coincide almost everywhere on [a, b], there exists ti £ {to — 5,to + 6) such that 
f{h) = gih)- Moreover, f < g, so 

f{to) < g{to) < g{ti) + e = /(ti) + e. (A.16) 

The continuity of / guarantees that for every e' > 0, there exists 5' > such that if |ti — to| < ^' ^ 



then f{t\) — e' < /(to) < /(^i) + ^' ■ Hence Equation (A.16) is rewritten as follows: 



/(to) < g{to) < /(to) + e+ e. 

As this inequality is valid for every e, e' > 0, g{to) = /(to) for every to E [a, b]. Thus f = g 
everywhere. □ 

A.l Lebesgue points for a real function 

After introducing the concept of measurable function and some properties of such functions, 
we state Lebesgue's differentiation theorem, which enables us to distinguish certain points for a 
measurable function. In the entire paper we consider the Lebesgue measure in M. See the book 
by Zaanen [75] for more details. 

Definition A. 8. A function / : [a, 6] C M — > M is measurable if the set {t G [a, 6] : /(t) > a} 
is measurable for every q G M. 

Definition A. 9. A function /: [a,b] ^ M is Lebesgue integrable over each Lebesgue measur- 
able set of finite measure if i^(x) = fdfi is well defined for every x £ [a, b]. 

Theorem A. 10. (Lebesgue's Differentiation Theorem f75^) Let fi be the Lebesgue mea- 
sure. If f : [a,b] ^ M. is a Lebesgue integrable function over every Lebesgue measurable set of 
finite measure, then for i'{x) = fdfi, 

Dv{x+) = Du{x^) = f{x) 

holds for fi-almost every x G [a, 5], where Di>{x+), Dv[x-) are the right and left derivatives of 
V respectively. 

The equality Dv{x-) = f{x) almost everywhere may be rewritten as follows for h > 

^ u(x-h)-u(x) ^, , , r-^ f{t)dt- r f{t)dt ^ 

lim ^ '-^ ^ = fix) a.e. ^ lim ^-^ — — , ^ ' = f{x) a.e. ^ 

r f(t)dt 

^ lim = fix) a.e. ^ / f{t)dt = hf{x) + o{h) a.e. 

Definition A. 11. If f : [a,b] M is a measurable function, x £ {a,b) is a Lebesgue point 
for f if, 

h^oJ^_h h 
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Remark A. 12. As Theorem A. 10 is true almost everywhere, the set of Lebesgue points for a 
measurable function has full measure. 

Remark A. 13. Observe that if n: / — > [/ is measurable and bounded, then it is integrable and 
the set of Lebesgue points for u has full measure. If / : [/ — > M is continuous, then f ou: I ^ M 
is integrable, and the intersection of Lebesgue points for u and f o u has full measure. 

Note: Assume we have a manifold Q, an open set [/ C M'^ and a continuous vector field X along 
the projection ir: Q xU ^ Q. If (7, u) : / = [a, 6] — > Q x t/, where 7 is absolutely continuous 
and u is measurable and bounded, then A o (7, li) : I ^ TQ is a measurable vector field along 
(7,ti), in the sense that in any coordinate system its coordinate functions are measurable. A 
point t £ (a, b) is a Lebesgue point for u if 

t 

X{-f{s),u{s))ds = hX{-f{t),u{t)) + o{h). (A.17) 

t~h 



The Lebesgue points for a vector field are useful in ^3.3, ^5.2 and in the following appendix to 
guarantee the differentiability of some curves, that is, the existence of its tangent vector. See 
EI] for more details about differential equations and measurability. 



B Time— dependent variational equations 



The variational equations give us an approach to how the integral curves of vector fields vary 
when the initial condition varies along a curve. These equations have a formulation on the 
tangent and the cotangent bundle. Here we are interested in studying the variational equations 
associated to time-dependent vector fields, and in proving some relationship between the solu- 
tions of variational equations on the tangent bundle and the ones on the cotangent bundle. See 
[50] for more details about these concepts. 



B.l Time— dependent vector fields 

As seen in ^ control systems are associated to a time-dependent vector field through a vector 
field along a projection. For I C M, a differentiable time- dependent vector field A is a mapping 
X: I X M ^ TM such that each {t,x) G I x M is assigned to a tangent vector X{t, x) in T^M. 
For every {s,x) £ I x M, the integral curve of X with initial condition (s,x) is denoted by 
^fs x) • '^{s,x) C I M and satisfies 



1- ^J,.)(«)=^- 



2. 



dt 



X 

(s,x) 



The domain of $^ ^-^ is denoted by J{s,x) C / because it depends on the initial condition for the 
integral curves. 

The time dependent flow or evolution operator of X is the mapping 

<^^: I X I X M — > M 

(^t,s,x) ^ci>^(t,,,x) = cl>j_^)(t) ^^-^^^ 

defined in a maximal open neighborhood V x AI oi Aj x M, where A/ is the diagonal of I x I, 
and satisfies 
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1. s,x) = X. 



2. 



dt 



{<^^{t,s,x)) = X{t,<!>^{t,s,x)). 



To obtain the original vector field through the evolution operator, the expression in the second 
assertion must be evaluated at s = t, 



dt 



<!>^{t,s,x)) 



X{t,x). 



There is a time-independent vector field on the manifold I x M associated to X and given 
by X{t,x) = d/dt\[t,x) + X{t,x). For {t,s,x) £ V x M, the flow of X is : / x / x M ^ 
I X M such that is the integral curve of X with initial condition (s, x) at time and 

^^{t, (s, x)) = (s+t, ^^{s+t, (s, x))). The theorems in differential equations about the existence 
and uniqueness of solutions guarantee the existence and uniqueness of the evolution operator 
defined maximally. 



For {t,s) eV C I X I, 



X 



M 

{t,s) 



(x) 



it) 



is a diffeomorphism on M satisfying 



(t,s) 



^ft,r) ° ^fr,s) r e I, such that (r, s), (t, r) G V. 



B.2 Complete lift 



Fromt the evolution operator of time-dependent vector fields in Equation (5.18), it is determined 
the evolution operator of a particular vector field on TM. 

Let Xt: M ^ TM be a vector field on M such that Xt{x) = X{t,x) for every t E I. The 
complete or tangent lift of Xf to TM is the time-dependent vector field Xf on TM satisfying 

= HM o TXt, 

where km is the canonical involution of TTM; that is, a mapping km '■ TTM — > TTM such that 
Id and ttm ° f^M = Ttm- See [50] for more details in the definition. Moreover, observe 



K 



M 



that Xf is a vector field that makes the following diagram commutative: 



TTM 



Ttm 



TM 



Ttm 




TXt 



TM 



TM 




TM- 

If {x,v) G TM, then TXt{x,v) = {x, Xt{x),nXt{v)) G T(^^^x,{x)){TM). 

Let {W, X*) be a local chart at x in M such that Xt = XI djdx^ where X\{x) = x) and 
X* G C°°{I X W). If (x*,f*) are the induced local coordinates in TM, then locally 



X^{t,x,v) =X\t,x) 



_d_ 
dx^ 



{x,v) 



dx^ ' dv^ 



{x,v) 



The equations satisfied by the integral curves of X"^ are called variational equations. 
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Proposition B.l. If X is a time-dependent vector field on M and is the evolution operator 
of X, then the map I x I x TM TM defined by 

^{t,s,ix,v)) = ($^(t,s,x),r,$J_,)(t;)) 

is the evolution operator of X'^ . 

Proof. We have to prove that 

^'(s,s, {x,v)) = {x,v); 



d_ 
dt 



[^{t, s, (x, v))) = X^{t, ^(t, s, (x, v))). 



The first item is proved easily, 

= (cl>-'^(s,s,a;),r, (7;)) = 

As for the second assertion, we use that ^^s) : ^ is a C°° diffeomorphism satisfying 



d_ 
dt 



X 



and we obtain 

d 
dt 



{^it,s,x,v)) 



($^(t,s,x)). 



dt 



x(t,$^(f,s,x)),(r, 



dt 



'^{t,s) n \[v, 



{X{t,'^^{t,s,x)), {T, {Xt{'^^{t,s,x)))) (v)) 

(x{t, s, x)), (r$x(i,,,,) (Xt) o r,.(c^^(t, s, x))) (i 

X{t,^^{t,s,x)),T^x^t,s,.) m (t,{^^^^){v))) = X^{t,^{t,s,x,v)). 



Hence, the evolution operator of X'^ is the complete lift of the evolution operator of X. The 
integral curves of X'^ are vector fields along the integral curves of X. □ 



B.2.1 About the geometric meaning of the complete lift 

The integral curves of X^ must be understood as the linear approximation of the integral curves 



of X when the initial condition varies along a curve in M. This idea appears in ^3.3 and ^ 

Let us explain the next figure. Given an integral curve of X with initial condition (s,x), 
we consider a curve a starting at the point x of the integral curve. Every point of a can be 
considered as the initial condition at time s for an integral curve of X. Thus the flow of X 
transports the curve a at a, different curve 6t point by point. The resultant curve is related with 
the complete lift of X as the following results prove. 
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Proposition B.2. Let X : IxM ^ TM be a time-dependent vector field with evolution operator 
and {s,x) £ I x M. For e > 0, let a: (-e,e) CR^ M be a curve such that a(0) = x £ 
M. For every t £ I, we define a curve 5t: (— e,e) M such that 

2. 6s{t) = cr(T), and 

3- m = ^fsAt)- 



Then5t{^)=T^^f.Aa{S))). 



-{s,x)\ 



Proof. 



^t(O) = (To<5t(r)) 



dT 



Tn ^ 



y^{s,a{r)) 



dT 



d^ 

dT 



d 

dT 



□ 



Corollary B.3. Let X be a time-dependent vector field on M . For x € M, x G T^M and for 

a small enough e > 0, let a: (— e,e) C M ^ M b e a C °° curve such that cr(0) = x £ M and 
(3"(0) = V. If 6t is the curve defined in Proposition \B.2 , then (5(.)(t): / — > TM, t h-> jt(r) is the 
integral curve of X^ with initial condition (s,(t(t)). 



Proof. The proof just comes from Propositions B.l and B.2 and the definition of the curve 
6t. □ 



B.3 Cotangent lift 

Given (t, s) £ I x I, the evolution operator is a diffeomorphism on TM and a hnear 

isomorphism on the fibers on TM, so it makes sense to consider its transpose and inverse, 
( '^^^ ^^)^^ = ^{t,s)- It is a hnear isomorphism on the fibers on T*M and satisfies ^[t,s) = ^{t,r)° 
^(r,s) foi' r £ I. Hence A: I x I x T*M T*M is the evolution operator of a time-dependent 
vector field on T*M, called the cotangent lift X'^* of X to T*M. 

Another vector field on T*M may be associated to X using the concepts in Hamiltonian 
formalism. For every t £ I, the Hamiltonian system ( M,uj, xA given by a symplectic manifold 
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{M,uj) and the Hamiltonian function Xt: T*M M, Xt{px) = ix{t,x)Px where px G T*M; has 
associated a Hamiltonian vector field Zt that satisfies Hamilton's equations izt^J = dXf. 
In local coordinates {x,p) for T*M, Z: I x T*M TT*M is given by 



Z{t,x,p) = X\t,x) 



dX^ , , d 



The equations satisfied by the integral curves of Z in the fibers arc the adjoint variational 
equations on the cotangent bundle. In the literature, they are sometimes called adjoint equations. 

Let us prove that both vector fields Z and X^* associated to X are the same. 

Proposition B.4. If X is a time-dependent vector field on M and is the evolution operator 
ofX, then A: Ixl x T*M T*M such that 



A{t,s,{x,p)) = {^''{t,s,x),i^^Tx^f^,^ 
is the evolution operator of Z. Thus Z = X^* . 
Proof. We have to prove that 



-1 



A(s,s,(x,p)) = {x,p), 

{A{t,s,{x,p))) = Z{t,A{t,s,{x,p))). 



d^ 
dt 



The first item is proved easily, 

A(s, s, {x,p)) = s, x), ( ^T^$J 3))"' {p) 

As : 7 X M — M is C°° , in local coordinates we have 



(a;,Id (p)) = {x,p). 



dt 



( ^Tx^ls)) = ( 



X 

{t,s) 



where both mappings go from T*M to T^x^j. (^sx))^' -'^^^ us prove the second assertion: 

d 



d^ 
dt 



{A{t,s,x,p)) 



dt 



(p) 



X{t, s,x)),(-[ ^Tx^l,)) o ( I L ( ^Tx^is) 



\dt\t ^{t,s) 



(^r^x^^(.) ((^^^^y 

= (^X(t,$^(i,s,x)),-(-r^x^^(,)(Xi)) (^l^-Tx^^^y {pYj^ = Z{t,A{t,s,x,p)). 
Hence, the evolution operator of Z is the cotangent lift of the evolution operator of X. Thus 
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B.4 A property for the complete and cotangent lift 



The previous propositions allow us to determine an invariant function along integral curves of 
X. 



Proposition B.5. Let X: IxM ^ TM be a time-dependent vector field and let X^ : IxTM 
TTM and X'^* : I x T*M ^ TT*M be the complete lift and cotangent lift of X, respectively. 
Ifj:I^Misan integral curve of X with initial condition {s,x), V: I ^ TM is the integral 
curve of X'^ with initial condition (s, {x,v)) where v G T^(g)M, and A: I ^ T*M is the integral 
curve of X^ with initial condition (s, {x,p)) where p E T*, ^M, then 



{A,V):I 
t 



7(s) 

M 

{A{t),V{t)) 



is constant along 7. 

Proof. If is the evolution operator of X, the evolution operators of X'^ and X'^* are 



{t,s,{x,p)) 



-1 



respectively, because of Propositions B.l and B.4 Hence 

-1 



(A(t),y(t)) 



X 



is))) {p),T,^ls){v) 



X 



o \ (v)) = {p,v) = constant. 



□ 



The tangent perturbation cone as an approximation of the 
reachable set 



In control systems, the reachable sets are useful to determine the accessibility and the con- 
trollability of the systems. In optimal control, the reachable set has a great importance for 
distinguishing the abnormal optimal curves from the normal ones |24| |52] . A key point in the 
proof of the Maximum Principle depends on the understanding of that linear approximation of 
the reachable set in a neighborhood of a point in the optimal curve. This interpretation of the 
tangent perturbation cone has been studied in [2J, but we will study it in a great and clear detail 
in this appendix. 

In the sequel, we explain why this interpretation of the tangent perturbation cone is feasible. 
Remember from §B.l that a time-dependent vector field on M has associated the evolution 
operator : I x I x M ^ M, {t,s,x) t-^ ^^{t,s,x) as defined in Equation (B.18). 



Proposition C.l. Let X , Y be time-dependent vector fields on M , then there exists a time-dependent 
vector field Z such that 



(CD 



and Z 



)*y. 
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Proof. For any initial time s, we define the diffeomorphism : I x M ^ I x M, {t, x) 
(t, such that ^'^{s,x) = {s,x). We look for a time-dependent vector field Z on M 

such that 

|.f+^(t,x) = ($f o|.f)(t,x). (C.19) 
This expression has been assumed true in [21, 152] for s = 0, but it has not been carefully proved. 



On the left-hand side of Equation (C.19) we have 

^f+^{t,x) = {t,^f+^it,x)) 

and the right-hand side is 



i'^f o |.f )(f, x) = cDf (t, cDf (t, x)) = (t, cDf {t, '^f it, x))) 



X, 



Thus Equation (C.19) is satisfied if and only if 

$f+^(t,x) = cl>f(t,$f(t,x)) = (cDf o|.f)(t,x) , 

or equivalently, 



Let us differentiate with respect to t the left-hand side of Equation (C.20), 

I^SJfW = + (0) = {X + Y){t, $f (t, $f (t, x))) . 



(C.20) 



(C.21) 



(C.22) 



The differentiation with respect to time of the right-hand side of Equation (C.20), for / in 

C°°(M), is 

l(^X(, ^Z(, ^^^ _ 1- m^sit + h, (f + h,xm- /(($f (t, (t, x)))) 



lim ■ 



+ 



if o <!>f){t + /i, cDf (t, x)) - (/ o $f $f (t, x)) 



T$f (i, x))f + c^f (t, cDf (t, x)))f. 



Hence 



d 



^^-(cDf (t, cDf (t, x))) = T^z(,,,)$(^^,)Z(t, cl>,^(t, x)) + a>f (t, cDf (t, x))) . 

From Equation ( C.22[ ) we have 

Xit, cDf (t, $f (t, x))) + cDf (t, cDf (t, x))) = T^z(i,,)Cl> cDf (t, x)) 

+ X(t,$f(t,cDf(f,x))) , 



that is, 



y((cl.f o|.f)(t,x)) =T^Z(,,,)C^(^,,)Z(t,cI>f(t,x)) . 
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Remember that the pushforward of a time-dependent vector field Z is another time-dependent 
vector field given by 

iryu /^X 



Then 



or equivalently, 



yo|.f o|.f = ($^_^)^z)o|.f o|.f , 



that is, y = , Z. 



Hence Z = (^^^^J^^y = (^>^^-,)*y. Now, going back to Equation (C.21) we have 



(C.23) 

□ 



Definition C.2. Let M be a manifold, U be a set in M'^ anfi X be a vector field along the 
projection tt: M x U ^ M. The reachable set from xq & M at time T ^ I is the set of 

points described by 

TZ{xq, T) = {x £ M I there exists (7, u): [a,b] ^ M x U such that 

7(t) = X(7(t), n(t)), 7(a) = xq, 7^ = x} . 

Once we know how to express the flow of a sum of vector fields as a composition of flows 
of different vector fields, we are going to show that all the integral curves used to construct the 



reachable set in Definition C.2 can be written as composition of flows associated with vector 



fields given by vectors in the tangent perturbation cone in Definition 3.11 



Each control system X G ^{t^) with the projection tt: M x U M is a time-dependent 
vector field X^^^ when the control is given. Consider the reference trajectory (7, u) to be an 
integral curve of X^^^ with initial condition xq at a. Take 7(ti) to be a reachable point from xq 
at time ti. Let us consider another control u: I ^ U and the integral curve of X^"> with initial 
condition xq at a denoted by 7. Then 7(^1) is another reachable point from xq at time ti. 



Let us see how to reach the point 7(^1) using Equation (C.23) 



-{ti,a) 



(ti,a) (ii,a) 




(C.24) 



Hence, from 7(^1) we can get every reachable point from xq at time ti through Equation (C.24) 
composing integral curves of the vector fields X^"} and ($ J^^"jp*(X^"} - X^"}) : I x M ^ TM, 
this latter with initial condition xq at a. 
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In fact this is true for any time r in [a, ti], that is, 



-1 



If we compose with the flow of X^'^\ we get a reachable point from xq at time ti because it is 
a concatenation of integral curves of the dynamical system, 



(C.25) 



Hence, from 7(ti) we can also get reachable points from xq at time ti through composition of 
integral curves of the vector fields Xi"> and (^^l^)* (^^"^ tlie latter with initial 

condition 7(a) at time a. 

On the other hand, the tangent perturbation cone at 7(^1) is given by the closure of the convex 
hull of all the tangent vectors f^^i r)) {-^^"'^'^^ 7(''")) ~ X^^\t, 7(t))) for every Lebesgue time 



r in [a, ti]. These vectors are related with the vector fields X^^J' through Equations (C.24) and 



(C.25). 



In this sense, we say that the tangent perturbation cone at 7(ti) is an approximation of the 
reachable set in a neighborhood of 7(^1). 



D Convex sets, cones and hyperplanes 

We study some properties satisfied by convex sets and cones; see |13| [65] for details. Unless 
otherwise stated, we suppose that all the sets are in a n-dimensional vector space E. We need 
to define the different kinds of cones and linear combinations used in this report. 

Definition D.l. A cone C with vertex at e E satisfies that ifv^C, then Xv e C for 
every A > 0. 

Definition D.2. Given a family of vectors V d E. 

1. A conic non negative combination of elements in V is a vector of the form XiVi + 
• • • + XrVr, with Ai > and Vi & V for all i £ {1, . . . , r}. 

2. The convex cone generated by V is the set of all conic non-negative combinations of 
vectors in V. 

3. An affine combination of elements in V is a vector of the form Xivi + • • • + XrVr, with 
Vi £ V , Xi £ for all i £ {1, . . . ,r} and J2l=i -^i = 1- 

4. A convex combination of elements in V is a vector of the form Xivi + • • • + XrVr, with 
Vi £ V , < Xi < 1 for all i £ {1, . . . ,r} and J2l=i = ^■ 

Remember that a set A C E is convex if, given two different elements in A, then any convex 
combination of them is contained in A. Thus, all the convex combination of elements in A are 
in A. 
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Definition D.3. The convex hull of a set A C E, conv(^), is the smallest convex subset 
containing A. 



Let us prove a characterization of the convex hull that will be useful. 

Proposition D.4. The convex hull of a set A is the set of the convex combinations of elements 
in A. 

Proof. Let us denote by C the set of all convex combinations of elements in A. First, we prove 

that C is a convex set. If .x, y arc in C, then they arc convex combinations of elements in A\ 
that is, X = Y!'i=\ ^i'^h y = YJi=i l^iWi, with Yh^i \i = 1, J2'i=i l^i = 1- Fo^' * ^ (0) 1)> consider 

sx+{l- s)y = s XiVi^ + (1 - s) l^iWi^ , 

that will be in C if the sum of the coefficients is equal to 1 and each of the coefficients lies in 
[0, 1]. Observe that s Yli=i + ~ ^) Yli=i IJ'i = s + {1 — s) = 1 and the other condition is 
satisfied trivially. As C is convex and contains A, the convex hull of ^ is a subset of C. 

Second, we prove that C C conv(^) by induction on the number of vectors in the convex 
combinations of elements in A. Trivially, when the convex combination is given by an element 
in A, it lies in the convex hull of A. 

Now, suppose that a convex combination of I — 1 elements of A is in conv(yl), and we prove 
that a convex combination of I elements of A is in conv(A). Let 

I i-i 



i=l i=l 

If^'~}//i = 0, then /i; = 1. By the first step of the induction, a; is in conv( A). IfX^|l}/Xi G (0,1], 
then fii G [0, 1) and we can rewrite x as 

x = {1- m) - niY^Vi + nivi. 



=1 



Observe that X]!=i/"j(l " w) ^ = (1 - " w) ^ = li and so I]i=i/"i(l " w) ^'"i is in 
conv(yl). By the first step of induction, vi is in conv(A). As (1 — ^i) + /i; = 1, x is in conv(^) 
because of the convexity of conv(^). Thus C C conv(A) and so C = conv(yl). □ 

Proposition D.5. Let C he a convex set. If C and int C are the topological closure and the 
interior of C, respectively, we have: 

(a) for every x G int C, if y & C, then (1 — X)x + Ay G int C for all A G [0, 1); 



(b) C = mt C; 

(c) the interior of C is empty if and only if the interior of C is empty; 

(d) int C = int C. 

Proof, (a) If X G int C, then there exists e^, > such that B{x, ex) C C, where B{x, Ex) denotes 
the open ball centered at x of radius ex. 

Observe that if y G C, for any e > 0, y G C + eS(0, 1) = {x + ez\x e C, z e B{0, 1)}. 
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For every A € [0, 1), we consider x\ = (1 — X)x + Ay. Let us compute the value of e\ such 
that XX + exB{0, 1) C C. 

XX + exB{0, 1) = (1 - X)x + Xy + exB{0, 1) 

C (1 - X)x + AC + Ae-B(0, 1) + exB{0, 1) = (1 - A)x + (Ae + ex)B{0, 1) + XC. 

If eA = (1 - A)e^ - Ae, then (1 - X)x + (Ae + ex)B{0, 1) C (1 - X)C and xx + e\B{0, 1) C C. 
For e > small enough, e^ is positive. Here we use the sum operation of convex sets, which 
is well-defined if the coefficients are positive (if Ci and C2 are convex sets, niCi + H2C2 is a 
convex set for all /ii,^2 > 0). 



(b) As int C C C, int C C C. 

On the other hand, each point in the closure of C can be approached along a line segment 
by points in the interior of C by (a). Thus C C int C. 

(c) As int C C int C, if int C is empty, then int C is empty. 

Conversely, if int C is empty, then by (6) C is empty. So C is empty and int C is also empty. 



(d) Trivially int C C int C. 

As the equality of the sets is true when they are empty because of (c), let us suppose that 
int C is not empty. If z G int C and take x £ int C, then there exists a small enough positive 
number 5 such that y = z + 6{z — x) G int C C C. 

Hence, 

1 <5 

y + ~i — F^- 



l + (5^ 1 + 6 
Note that 

1 S I 6 

0<^ ? < 1' 0<^ ? < 1> ? + ^ F = l- 

1+6 1+6 1+6 1+6 

Asy eC, X eint C and 1/(1 + 6) lies in (0, 1). By (a), z G int C. □ 

Remark D.6. Consequently, if C is convex and dense, then C is the whole space. 

The following paragraphs introduce elements playing an important role in the proof of Pon- 
tryagin's Maximum Principle in ^and ^ 

Definition D.7. Let C be a cone with vertex at ^ E. A supporting hyperplane to C at 

is a hyperplane such that C is contained in one of the half-spaces defined by the hyperplane. 

Remark D.8. In a geometric framework, we will define a hyperplane in E as the kernel of a 
nonzero 1-form a in the dual space E* . Then the hyperplane Pa associated to a is ker q. Hence 
the supporting hyperplane to C at is a hyperplane Pa such that a{v) < for all v £ C. A 
supporting hyperplane to C at is not necessarily unique. 

From now on, we consider that all the cones have vertex at 0. 

Definition D.9. Let C be a cone, the polar of C is 

C* = {aeE*\ a{v) < , V u G C}. 



Note that the polar of a cone is a closed and convex cone in E* . 
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Definition D.IO. Let C he a cone, the set 

C** = {weE \ a{w) < , Va G C7*} 
is called the polar of the polar of C . 

Observe that C C C** . The fohowing lemma is used in the proof of the existence of a 
supporting hyperplane to a cone with vertex at 0. 

Lemma D.ll. The cone C is closed and convex if and only if C** = C. 
Proof. Observe that 

C** = {weE \ a{w) < , Va G C*} = {w e E \ a{w) < 0}. 

Then C** = conv(C), because of Theorem 6.20 in Rockafellar [65]: the closure of the convex 
hull of a set is the intersection of all the closed half-spaces containing the set. Now, the result 
is immediate. □ 

The following proposition guarantees the existence of a supporting hyperplane to a cone with 
vertex at 0. This result is used throughout the proof of Pontryagin's Maximum Principle in ^ 
and g 

Proposition D.12. If C is a convex and closed cone that is not the whole space, then there 
exists a supporting hyperplane to C at 0. 



Proof. If there is no supporting hyperplane containing the cone in one of the two half-spaces, 
then for all a £ E* there exist vi, V2 G C with a{vi) < and 0(^2) > 0. Thus C* = {0} and 
C** = E. Then, by Lemma D.ll, C = C** = E m. contradiction with the hypothesis on C. □ 



Corollary D.13. If C is a convex cone that is not the whole space, then there exists a supporting 
hyperplane to C at 0. 



Proof. If C ^ E, then C ^ E hy Proposition D.5 (d). Hence, by Proposition D.12, there exists 
a supporting hyperplane to C which is also a supporting hyperplane to C. □ 

Definition D.14. Let Ci and C2 he cones with common vertex 0. They are separated if there 
exists a hyperplane P such that each cone lies in a different closed half-space defined hy P. This 
P is called a separating hyperplane of C\ and C2. 



A point x is a relative interior point of a set C, if x G C and there exists a neighbourhood 
V of X such that V n aff(y) C V . Then, a useful characterization of separated convex cones is 
the following: 

Proposition D.15. The convex cones Ci and C2, with common vertex 0, are separated if and 
only if one of the two following conditions are satisfied: 

(1) there exists a hyperplane containing hoth Ci and C2, 

(2) there is no point that is a relative interior point of hoth Ci and C2. 
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Proof. =^ If Ci and C2 are separated, then there exists a separating hyperplane Pa such that 

a{vi)<0 Vwi G Ci, a(v2) > Vw2 G C2. 

If a{vi) = for ah Vi £ Ci and i = 1,2, then we are in the first case. 

If some Vi £ Ci satisfies the strict inequahty, then both sets do not he in the hyperplane Pa ■ 
They he in a different closed half-space. If the convex cones intersect, the intersection lies in 
the boundary of the cones and in the hyperplane. Hence, there is no point that is a relative 
interior point of both Ci and C2. 

<^ First, we are going to prove that if (1) is true, then Ci and C2 are separated. As there 
exists a hyperplane determined by a such that a{vi) = for all Vi G Ci, a determines a separating 
hyperplane of Ci and C2. 

Now, we are going to prove that if (2) is true, then Ci and C2 are separated. As Ci and C2 
are convex cones, 

Cl - C2 = {U e E \ U = Vi - V2, Vl e Cl, V2 G C2} 

is a convex cone. Since there is no relative interior point of both Ci and C2, does not lie 
in Cl — C2 . By Corollary D.13| there exists a supporting hyperplane Pa to Ci — C2 such that 



a{vi — V2) < 0, that is, a{vi) < a{v2), for all vi £ Ci, V2 £ C2. 

Observe that a supporting hyperplane to Ci — C2 is a supporting hyperplane to Ci, because, 
taking V2 = 0, a{vi) < a{v2) = for all vi £ Ci. 

As (9(Ci — C2) n Cl C dCi, we consider a supporting hyperplane Pa to Ci — C2 such that 
a{vi) = for some vi G dCi. Hence a{v2) > a{vi) = for all V2 G C2. As a{vi) < for all 
vi G Cl, a determines a separating hyperplane of Ci and C2. □ 

This proposition gives us necessary and sufficient conditions for the existence of a separating 
hyperplane of two convex cones with common vertex. Observe that a separating hyperplane of 
two cones with common vertex is also a supporting hyperplane to each cone at the vertex. 

Corollary D.16. If the convex cones Ci and C2 with common vertex are not separated, then 
-E/ = Cl — C2. 



Proof. If the cones are not separated, by Proposition D.15 there exists no any hyperplane con- 
taining both and the intersection of their relative interior is not empty. 

Let us suppose that the convex cone Ci — C2 7^ E. Then, by Corollary |D. 13 there exists a 



supporting hyperplane determined by A at the vertex such that X{v) > for every v in Ci — C2. 

Because of the definition of cones, if f 1 G Ci, then vi £ Ci — C2 and A(t;i) > 0. Analogously, 
if V2 G C2, then -V2 G Ci - C2 and X{-V2) > 0, that is, X{v2) < 0. □ 



E One corollary of Brouwer Fixed— Point Theorem 

From the statement of Brouwer Fixed-point Theorem, it is possible to prove a corollary in [53] 



useful for the proof of Proposition 3.12 



Theorem E.l. (Brouwer Fixed point Theorem) Let be the closed unit ball in W^. Any 
continuous function G : B1 B^ has a fixed point. 

Corollary E.2. Let g: B^ M" be a continuous map. Let P be an interior point of Bf. If 
\\g{x) — x\\ < \\x — P\\ for every x in the boundary dBf, then the image g{B'^) covers P. 
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Proof. Without loss of generality, we assume that P is the origin of M". Consider the mapping 
g as a continuous vector field on the unit ball B^. 

As \\g{x) — x\\ < \\x\\, we are going to show that g{x) makes an acute angle with the outward 
ray from the origin through x for every x £ dB^. Let us consider the equality 

— + — x||^ = \\y — xW"^ + 2{y — z, X — z) , 

and take y = g{x) and 2 = 0. Then 

2{g{x),x) = Ux)f + \\xf - \\g{x) - xf > Ux)f + \\xf - \\xf = Ux)f > 0. 

Thus g{x) makes an acute angle with x. So g{x) has an outward radial component at every point 
X £ dBf. The vector —g{x) has a negative radial component. For a sufficiently small positive 



number a the function x ^ x — ag{x) goes from B^ to B^. By Theorem E.l there exists a 
fixed point xq such that xq = xq — ag{xQ), then ag{xQ) = and g{xo) = since a S M^. As g 
is continuous and ^(xo) = 0, the image of a neighbourhood of xq covers the origin. □ 
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